Moving picture encoding device, moving picture decoding device, moving picture encoding method, moving picture decoding method, program, and computer readable recording medium storing program

ABSTRACT

A moving picture encoding device for encoding a moving picture constituted of a time sequence of frame pictures by motion compensation, the device including: a reference picture generation section configured to generate a reference picture subjected to predetermined picture processing from a reference frame picture, in accordance with an encoding condition of a predetermined area to be encoded; and a motion compensation section configured to calculate a motion compensation value for the predetermined area to be encoded, by using a generated reference picture subjected to the predetermined picture processing.

CROSS-REFERENCE TO RELATED APPLICATION

The present divisional application claims the benefit of priority under35 U.S.C. §120 to application Ser. No. 10/496,017, filed Jun. 1, 2004,which is the National Stage of PCT/JP02/12556, filed on Nov. 29, 2002and under 35 U.S.C. § 119 from Japanese applications Nos. 2001-367940,filed on Nov. 30, 2001 and 2002-129434 filed on Apr. 30, 2002, theentire contents of these applications are incorporated herein byreference.

TECHNICAL FIELD

The present invention relates to a moving picture encoding device, amoving picture decoding device, a moving picture encoding method, amoving picture decoding method, a program, and a computer readablerecording medium which has stored the program.

BACKGROUND ART

As an example of a conventional moving picture encoding system, a movingpicture encoding device and a moving picture decoding device will bedescribed based on an “H. 26L encoding system” described in “ITU-T SG16VCEG-M81, H. 26L Test Model Long Term Number 8 (TML-8)”. FIG. 1 shows aconfiguration of the aforementioned moving picture encoding device 20,and FIG. 2 shows a configuration of the aforementioned moving picturedecoding device 50.

The moving picture encoding device shown in FIG. 1 reduces a redundancypresent in a time direction by motion compensation inter-frameprediction, and further reduces a redundancy left in a space directionby orthogonal transformation, so as to execute information compressionof a moving picture (an input video signal). FIG. 3 shows an explanatorydiagram of the motion compensation inter-frame prediction.

Hereinafter, an operation of the moving picture encoding device 20 shownin FIG. 1 will be described with reference to these drawings.

An input video signal 1 is constituted of a time sequence of framepictures. Here, it is assumed that the frame picture to be encoded isdivided into square rectangular areas (macro-blocks) of 16×16 pixels,and an encoding process in the moving picture encoding device 20 and adecoding process in the moving picture decoding device 50 are carriedout by units of these macro-blocks. Additionally, the frame picturewhich is divided into the macro-block units is defined as “a framepicture signal 2”.

According to the “H. 26L encoding system”, what are available as“prediction modes” are an “INTRA prediction mode” for executing spaceprediction which uses pixel values of encoded neighboring areas on thesame frame picture (e.g., pixel values adjacent to the upper and leftsides of a frame picture signal 2 to be encoded), and a plurality of“INTER prediction modes” for executing motion compensation inter-frameprediction which uses encoded frame pictures (reference frame pictures5) different with time.

The “H. 26L encoding system” is configured such that efficientinformation compression can be carried out by switching the “predictionmode” by a macro-block unit, in accordance with a local nature of theinput video signal 1.

The “motion compensation inter-frame prediction” is a technology forsearching an picture signal pattern similar to an picture signal patternin the frame picture signal 2 within a predetermined search range of areference frame picture 5, for detecting a spatial displacement amountbetween both picture signal patterns as a “motion vector 3”, and forencoding and transmitting “motion compensation related information”containing the “motion vector 3,” the “prediction mode” and a “referenceframe number,” as well as a “predicted residual signal 9” calculated inaccordance with the motion vector 3.

According to the “H. 26L encoding system”, as shown in FIG. 3, 7 kindsof “INTER prediction modes” are available. More exactly, in addition tothese INTER prediction modes, available is a “skip mode” useful when avideo is static, i.e., a prediction mode for directly copying a pixel inthe same position of the reference frame picture 5 (the encoded framepicture) as it is.

As shown in FIG. 3, the motion vector 3 is detected by a unit of 16×16pixels on a “mode 1”, by a unit of 8×16 pixels on a “mode 2”, by a unitof 16×8 pixels on a “mode 3”, by a unit of 8×8 pixels on a “mode 4”, bya unit of 4×8 pixels on a “mode 5”, by a unit of 8×4 pixels on a “mode6”, and by a unit of 4×4 pixels on a “mode 7”.

That is, these 7 kinds of prediction modes enable subdivision of motiondetection units in the macro-block, and are disposed for the purpose ofaccurately grasping various motions that can be present in themacro-block.

First, an input section 31 transmits the frame picture signal 2 to amotion detection section 32 and a space prediction section 35.

Subsequently, the motion detection section 32 detects the number ofmotion vectors 3 corresponding to a predetermined prediction mode 4 forthe received frame picture signal 2, by referring to the reference framepicture 5 sent from a frame memory 34.

Meanwhile, the space prediction section 35 carries out space predictionthat uses pixel values of encoded neighboring areas on the same framepicture sent from the frame memory 34. The space prediction section 35may execute space prediction by a plurality of methods.

Second, the motion detection section 32 transmits motion vectors 3detected for all the “INTER prediction modes” shown in FIG. 3, and theprediction modes (e.g., modes 1 to 7) 4 corresponding to the motionvectors 3, to a motion compensation section 33.

Subsequently, the motion compensation section 33 generates a predictedpicture signal (a macro-block unit) 6, by motion compensation which usesthe reference frame picture 5 sent from the frame memory 34 and acombination of the plurality of motion vectors 3 and prediction modes 4sent from the motion detection section 32.

Third, the motion compensation section 33 transmits informationregarding the predicted picture signal 6 generated by the motioncompensation, the prediction mode 4, the motion vectors 3 and encodingefficiency, to a prediction mode determining section 36. On the otherhand, the space prediction section 35 transmits information regarding apredicted picture signal 7 generated by space prediction, the predictionmode (if there are a plurality of kinds of space prediction) 4 andencoding efficiency, to the prediction mode determining section 36.

Fourth, the prediction mode determining section 36 evaluates all the“INTER prediction modes” shown in FIG. 3 by a macro-block unit, so as toselect an “INTER prediction mode” which is determined to be highest inencoding efficiency.

Additionally, the prediction mode determining section 36 similarlyevaluates the “INTRA prediction modes”, and selects the “INTRAprediction mode” if the “INTRA prediction mode” is higher in encodingefficiency than the “INTER prediction mode”.

Then, the prediction mode determining section 36 transmits a predictedpicture signal (a macro-block unit) 8 generated by the selectedprediction mode 4, to a subtracter 37.

Additionally, when the “INTER prediction mode” is selected as theprediction mode 4, the prediction mode determining section 36 transmits“motion compensation related information” containing the number (up to16 per macro-block) of motion vectors 3 or the like set on the selected“INTER prediction mode”, to a variable length encoding section 40. Onthe other hand, when the “INTRA prediction mode” is selected as theprediction mode 4, the prediction mode determining section 36 transmitsno motion vectors 3.

Fifth, an orthogonal transformation section 38 generates an orthogonaltransformation coefficient 10, by applying orthogonal transformation toa difference value (a predicted residual signal 9) between the framepicture signal 2 and the predicted picture signal 8 sent from thesubtracter 37.

Sixth, a quantization section 39 generates a quantized orthogonaltransformation coefficient 11, by quantizing the orthogonaltransformation coefficient 10 sent from the orthogonal transformationsection 38.

Seventh, the variable length encoding section 40 carries out entropyencoding for the quantized orthogonal transformation coefficient 11 sentfrom the quantization section 39 and the prediction mode 4 (and motionvectors 3) sent from the prediction mode determining section 36, so asto multiplex them into a compressed stream 12.

The variable length encoding section 40 may transmit the compressedstream 12 to a moving picture decoding device 50 by a macro-block unit,or transmit the compressed stream 12 by a frame picture unit.

Additionally, an inverse quantization section 41 generates an orthogonaltransformation coefficient 13, by carrying out inverse quantization forthe quantized orthogonal transformation coefficient 11 sent from thequantization section 39. Then, an inverse orthogonal transformationsection 42 generates a predicted residual signal 14, by carrying outinverse orthogonal transformation for the orthogonal transformationcoefficient 13 sent from the inverse quantization section 41.

Next, at an adder 43, the predicted residual signal 14 sent from theinverse orthogonal transformation section 42 and the predicted picturesignal 8 sent from the prediction mode determining section 36 are addedtogether to generate a frame picture signal 15.

This frame picture signal 15 of a macro-block unit is stored in theframe memory 34. In the frame memory 34, there have been stored areference frame picture 5 of a frame picture unit used for a subsequentencoding process, and information (a pixel value or a motion vector) ofan encoded macro-block of a frame picture which is currently beingencoded.

Next, an operation of the moving picture decoding device 10 shown inFIG. 2 will be described.

First, after reception of the compressed stream 12, a variable lengthdecoding section 71 detects a synchronous word indicating a head of eachframe, and restores the motion vector 3, the prediction mode 4 and thequantized orthogonal transformation coefficient 11 for each macro-blockunit.

Then, the variable length decoding section 71 transmits the quantizedorthogonal transformation coefficient 11 to an inverse quantizationsection 76, and transmits the prediction mode 4 to a switch 75.

Additionally, the variable length decoding section 71 transmits themotion vector 3 and the prediction mode 4 to a motion compensationsection 72 when the prediction mode 4 is an “INTER prediction mode”, andtransmits the prediction mode 4 to a space prediction section 74 whenthe prediction mode 4 is an “INTRA prediction mode”.

Next, when the prediction mode 4 is the “INTER prediction mode”, themotion compensation section 72 generates a predicted picture signal 6,by using the motion vector 3 and the prediction mode 4 sent from thevariable length decoding section 71 and referring to a reference framepicture 5 sent from a frame memory 73.

On the other hand, when the prediction mode 4 is the “INTRA predictionmode”, the space prediction section 74 generates a predicted picturesignal 7, by referring to an encoded picture signal of a neighboringarea sent from the frame memory 73.

Next, the switch 75 chooses any one of the predicted picture signals 6and 7, in accordance with the prediction mode 4 sent from the variablelength decoding section 71, so sa to determine a predicted picturesignal 8.

Meanwhile, the quantized orthogonal transformation coefficient 11decoded by the variable length decoding section 71 is subjected toinverse quantization by the inverse quantization section 76, so as to berestored as an orthogonal transformation coefficient 10. And theorthogonal transformation coefficient 10 is subjected to inverseorthogonal transformation by an inverse orthogonal transformationsection 77, so as to be restored as a predicted residual signal 9.

Then, at an adder 78, the predicted picture signal 8 sent from theswitch 75 and the predicted residual signal 9 sent from the inverseorthogonal transformation section 77 are added together, and the framepicture signal 2 is thereby restored to be sent to an output section 80.The output section 80 outputs the signal to a display device (not shown)with predetermined timing, so as to reproduce an output video signal (amoving picture) 1A.

Additionally, the restored frame picture signal 2 is stored in the framememory 73, so as to be used for a decoding process thereafter.

In the “TML-8”, motion compensation which uses a concept of a “funnyposition” is realized. FIG. 4 shows this “funny position” together withan integer picture position, a ½ picture position, and a ¼ pictureposition. Incidentally, in the “TML-8”, motion compensation of ¼ pixelaccuracy is realized.

In FIG. 4, it is assumed that the motion vector 3 detected by the motiondetection section 32 indicates an integer pixel position (the pixelposition of (1 pixel, 1 pixel)) “D” in the reference frame picture 5 inrelation to an integer pixel position “A” in the frame picture signal 2to be encoded. In this case, a pixel value of the pixel position “D” inthe reference frame picture 5 becomes a “motion compensation value” inrelation to the pixel position “A” in the frame picture signal 2 to beencoded.

Next, it is assumed that the motion vector 3 indicates a ½ pixelposition (the pixel position of (½ pixel, ½ pixel)) “E” in the referenceframe picture 5 in relation to the integer pixel position “A” in theframe picture signal 2 to be encoded. In this case, an interpolationvalue obtained by independently operating 6 tap filters (1, −5, 20, 20,−5, 1)/32 vertically and horizontally for the pixel value of the integerpixel position in the reference frame picture 5 becomes a “motioncompensation value” in relation to the pixel position “A” in the framepicture signal 2 to be encoded.

Next, it is assumed that the motion vector 3 indicates a ¼ pixelposition (a pixel position of (¼ pixel, ¼ pixel)) “F” or “G” in thereference frame picture 5 in relation to the integer pixel position “A”in the frame picture signal 2 to be encoded. In this case, a linearinterpolation value of a pixel value of a neighboring integer pixelposition and a pixel value of a neighboring ½ pixel position 5 becomes a“motion compensation value” in relation to the pixel position “A” in theframe picture signal 2 to be encoded.

For example, when the motion vector 3 indicates the pixel position “F”in the reference frame picture 5 in relation to the pixel position “A”in the frame picture signal 2 to be encoded, an average of 4 points ofthe pixel value of the neighboring integer pixel position and the pixelvalues of the neighboring ½ pixel positions which surround the pixelposition “F” becomes a “motion compensation value” in relation to thepixel position A in the frame picture signal 2 to be encoded.

Additionally, when the motion vector 3 indicates the pixel position “G”in the reference frame picture 5 in relation to the integer pixelposition A in the frame picture signal 2 to be encoded, an average of 2points of the pixel values of the ½ pixel positions which horizontallysandwich the pixel position “G” becomes a “motion compensation value” inrelation to the pixel position A in the frame picture signal 2 to beencoded.

Further, when the motion vector indicates a pixel position of (N+¾pixel, M+¾ pixel: N and M are given integers) in the reference framepicture 5 in relation to an integer pixel position in the frame picturesignal 2 to be encoded, a “motion compensation value” in relation to theinteger pixel position in the frame picture signal 2 to be encodedbecomes an average of a pixel value of (N, M), a pixel value of (N,M+1), a pixel value of (N+1, M) and a pixel value of (N+1, M+1) in thereference frame picture 5. Here, (N+¾ pixel, M+¾ pixel: N and M aregiven integers) in the reference frame picture 5 is the aforementioned“funny position”.

For example, when the motion vector 3 indicates a pixel position “H”(i.e., a “funny position”) in the reference frame picture 5 in relationto the integer pixel position “A” in the frame picture signal 2 to beencoded, a “motion compensation value” in relation to the pixel position“A” in the frame picture signal 2 to be encoded is not a valuecalculated in the aforementioned case of the ¼ pixel position (e.g., thepixel position “F”), but a value obtained by calculation of (A+B+C+D)/4.

As described above, in the “H. 26L encoding system”, many “INTERprediction modes” are available to enable elaborate motion compensation.Additionally, motion compensation based on the integer pixel position,the ½ pixel position, the ¼ pixel position and the funny position areavailable. By the foregoing configuration, while a configuration forprediction is elaborated, a mechanism is introduced to prevent breakageof the predicted picture signal 8 even if a frame picture signal 2 whoseprediction would not be fulfilled is inputted.

The calculation of ¼ picture accuracy is carried out by linearinterpolation of the pixel values of the neighboring pixel positions.Thus, a low-pass type operation is provided in a frequency space, so asto generate a smoothed predicted picture signal 6.

Additionally, when motion compensation based on the funny position isused, a “motion compensation value” is calculated based on an average ofpixel values of 4 neighboring integer pixel positions, so as to generatea further smoothed predicted picture signal. If Gaussian noise issuperimposed on the predicted picture signal, the smoothing has aneffect of reducing a prediction error when this noise component islarge.

Thus, in the “H. 26L encoding system” defined by the “TML-8”, if noiseis superimposed on the reference frame picture 5, or if many high-passcomponents are contained in the reference frame picture 5 and an errorin prediction is flagrant, encoding efficiency is improved by using thecalculation of ¼ pixel accuracy and the motion compensation based on thefunny position.

However, the following problems conceivably occur in the conventional“H. 26L encoding system”.

First, when a pixel position in the frame picture signal 2 to be encodedhas a motion vector which indicates a pixel position (N+¾ pixel, M+¾pixel: N and M are given integers) equal to the “funny position,” acalculated “motion compensation value” is always subjected to strongsmoothing, and especially it has been a problem that elaborate motioncompensation is hindered at a high rate (a first problem).

That is, in the conventional “H. 26L encoding system”, the “funnyposition” is defined by an absolute value of the motion vector 3. Thus,as shown in FIG. 5, for example, when blocks A, B, C, D and E move inparallel on a right lower side (¾ pixel, ¾ pixel), smoothed motioncompensation is carried out based on a motion vector MV=(MVx, MVy)=(¾,¾). Alternatively, motion compensation is carried out by feeding amotion vector different from real motion based on a motion vectorMV=(MVx, MVy)=(½, ¾) or (¾, 1). Here, MVx indicates an X element of themotion vector, and MVy indicates a Y element of the motion vector.

Specifically, as shown in FIG. 5, in the conventional “H. 26L encodingsystem”, when a block to be encoded is E, and a motion vector MV of theblock E is (MVx_(E), MVy_(E)), an area expressed by “MVx_(E)%4=3” and“MVy_(E)%4=3” always is a “funny position,” and a smoothed pixel valueis chosen as a “motion compensation value” for the block E. Here, “%” isa quotient remainder calculation symbol, and a unit for expressing themotion vector MV is a ¼ pixel.

Thus, in the “H. 26L encoding system”, since the motion vector (¾, ¾)indicates a smoothed pixel value present in a real (½, ½) pixelposition, it has been a problem that the expressing of a pixel value ofa pixel position (N+¾ pixel, M+¾ pixel: N and M are given integers)equal to the “funny position” is hindered from being expressed.

Second, in the generating of a predicted picture signal by ¼ pixelaccuracy, effects of elaboration of prediction and smoothing ofprediction are respectively expected at a high rate and a low rate.However, with regard to the smoothing of prediction at the low rate,motion compensation of ¼ pixel accuracy is not necessary but realizationof motion compensation of ½ pixel accuracy is sufficient. Consequently,it has been a problem that detection of the motion vector of ¼ pixelaccuracy which occupies a half of a parameter space of the motion vectorfor smoothing prediction is redundant.

The present invention, therefore, has been made with the foregoingproblems in mind, and an object of the invention is to express apredicted picture signal with lighter overheads, and to provide motioncompensation of different degrees of pixel accuracy.

DISCLOSURE OF THE INVENTION

A first feature of the present invention is summarized as a movingpicture encoding device for encoding a moving picture constituted of atime sequence of frame pictures by motion compensation. The movingpicture encoding device includes a motion vector detection sectionconfigured to detect a motion vector of a predetermined area to beencoded in the frame picture; a prediction section configured to predictthe motion vector of the predetermined area to be encoded by using anencoded motion vector of a predetermined area in the frame picture; adetermination section configured to determine whether or not the motionvector detected by the motion vector detection section is apredetermined motion vector set in accordance with the motion vectorpredicted by the prediction section; and a switching section configuredto switch a method of calculating a motion compensation value for thepredetermined area to be encoded depending on whether or not the motionvector detected by the motion vector detection section is thepredetermined motion vector.

A second feature of the present invention is summarized as a movingpicture decoding device for decoding a moving picture constituted of atime sequence of frame pictures by motion compensation. The movingpicture decoding device includes a motion vector decoding sectionconfigured to decode a motion vector of a predetermined area to bedecoded in the frame picture; a prediction section configured to predictthe motion vector of the predetermined area to be decoded by using adecoded motion vector of a predetermined area in the frame picture; adetermination section configured to determine whether or not the motionvector decoded by the motion vector decoding section is a predeterminedmotion vector set in accordance with the motion vector predicted by theprediction section; and a switching section configured to switch amethod of calculating a motion compensation value for the predeterminedarea to be decoded depending on whether or not the motion vector decodedby the motion vector decoding section is the predetermined motionvector.

A third feature of the present invention is summarized as a movingpicture encoding method for encoding a moving picture constituted of atime sequence of frame pictures by motion compensation. The movingpicture encoding method includes a step A of detecting a motion vectorof a predetermined area to be encoded in the frame picture; a step B ofpredicting the motion vector of the predetermined area to be encoded byusing an encoded motion vector of a predetermined area in the framepicture; a step C of determining whether or not the motion vectordetected in the step A is a predetermined motion vector set inaccordance with the motion vector predicted in the step B; and a step Dof switching a method of calculating a motion compensation value of thepredetermined area to be encoded depending on whether or not the motionvector detected in the step A is the predetermined motion vector.

A fourth feature of the present invention is summarized as a movingpicture decoding method for decoding a moving picture constituted of atime sequence of frame pictures by motion compensation. The movingpicture decoding method includes a step A of decoding a motion vector ofa predetermined area to be decoded in the frame picture; a step B ofpredicting the motion vector of the predetermined area to be decoded byusing a decoded motion vector of a predetermined area in the framepicture; a step C of determining whether or not the motion vectordecoded in the step A is a predetermined motion vector set in accordancewith the motion vector predicted in the step B; and a step D ofswitching a method of calculating a motion compensation value of thepredetermined area to be decoded depending on whether or not the motionvector decoded in the step A is the predetermined motion vector.

A fifth feature of the present invention is summarized as a program forcausing a computer to function as a moving picture encoding device forencoding a moving picture constituted of a time sequence of framepictures by motion compensation. The moving picture encoding deviceincludes a motion vector detection section configured to detect a motionvector of a predetermined area to be encoded in the frame picture; aprediction section configured to predict the motion vector of thepredetermined area to be encoded by using an encoded motion vector of apredetermined area in the frame picture; a determination sectionconfigured to determine whether or not the motion vector detected by themotion vector detection section is a predetermined motion vector set inaccordance with the motion vector predicted by the prediction section;and a switching section configured to switch a method of calculating amotion compensation value of the predetermined area to be encodeddepending on whether or not the motion vector detected by the motionvector detection section is the predetermined motion vector.

A sixth feature of the present invention is summarized as a program forcausing a computer to function as a moving picture decoding device fordecoding a moving picture constituted of a time sequence of framepictures by motion compensation. The moving picture decoding deviceincludes a motion vector decoding section configured to decode a motionvector of a predetermined area to be decoded in the frame picture; aprediction section configured to predict the motion vector of thepredetermined area to be decoded by using a decoded motion vector of apredetermined area in the frame picture; a determination sectionconfigured to determine whether or not the motion vector decoded by themotion vector decoding section is a predetermined motion vector set inaccordance with the motion vector predicted by the prediction section;and a switching section configured to switch a method of calculating amotion compensation value of the predetermined area to be decodeddepending on whether nor not the motion vector decoded by the motionvector decoding section is the predetermined motion vector.

A seventh feature of the present invention is summarized as a computerreadable recording medium which stores a program for causing a computerto function as a moving picture encoding device for encoding a movingpicture constituted of a time sequence of frame pictures by motioncompensation. The moving picture encoding device includes a motionvector detection section configured to detect a motion vector of apredetermined area to be encoded in the frame picture; a predictionsection configured to predict the motion vector of the predeterminedarea to be encoded by using an encoded motion vector of a predeterminedarea in the frame picture; a determination section configured todetermine whether or not the motion vector detected by the motion vectordetection section is a predetermined motion vector set in accordancewith the motion vector predicted by the prediction section; and aswitching section configured to switch a method of calculating a motioncompensation value of the predetermined area to be encoded depending onwhether or not the motion vector detected by the motion vector detectionsection is the predetermined motion vector.

In the seventh feature of the invention, the predetermined motion vectoris preferably set to be different from the motion vector predicted bythe prediction section.

Additionally, in the seventh feature of the invention, when differenceinformation between the motion vector predicted by the predictionsection and the motion vector detected by the motion vector detectionsection is a predetermined value, the determination section preferablydetermines that the motion vector detected by the motion vectordetection section is the predetermined motion vector.

An eighth feature of the present invention is summarized as a computerreadable recording medium which stores a program for causing a computerto function as a moving picture decoding device for decoding a movingpicture constituted of a time sequence of frame pictures by motioncompensation. The moving picture decoding device includes a motionvector decoding section configured to decode a motion vector of apredetermined area to be decoded in the frame picture; a predictionsection configured to predict the motion vector of the predeterminedarea to be decoded by using a decoded motion vector of a predeterminedarea in the frame picture; a determination section configured todetermine whether or not the motion vector decoded by the motion vectordecoding section is a predetermined motion vector set in accordance withthe motion vector predicted by the prediction section; and a switchingsection configured to switch a method of calculating a motioncompensation value of the predetermined area to be decoded depending onwhether nor not the motion vector decoded by the motion vector decodingsection is the predetermined motion vector.

In the eighth feature of the invention, the predetermined motion vectoris preferably set to be different from the motion vector predicted bythe prediction section.

Additionally, in the eighth feature of the invention, when differenceinformation between the motion vector predicted by the predictionsection and the motion vector decoded by the motion vector decodingsection is a predetermined value, the determination section preferablydetermines that the motion vector decoded by the motion vector decodingsection is the predetermined motion vector.

A ninth feature of the present invention is summarized as a movingpicture encoding device for encoding a moving picture constituted of atime sequence of frame pictures by motion compensation. The movingpicture encoding device includes a reference picture generation sectionconfigured to generate a plurality of different reference pictures byexecuting a plurality of different picture processing on a referenceframe picture; a motion compensation section configured to calculate amotion compensation value for a predetermined area to be encoded byusing the generated reference picture; and a transmission sectionconfigured to transmit a combination of information regarding thereference picture used for calculating the motion compensation value andinformation indicating the motion compensation value. The informationregarding the reference picture is a combination of identificationinformation of the reference frame picture and information indicatingthe picture processing.

A tenth feature of the present invention is summarized as a movingpicture encoding device for encoding a moving picture constituted of atime sequence of frame pictures by motion compensation. The movingpicture encoding device includes a reference picture generation sectionconfigured to generate a reference picture subjected to predeterminedpicture processing from a reference frame picture in accordance with anencoding condition of a predetermined area to be encoded; and a motioncompensation section configured to calculate a motion compensation valuefor the predetermined area to be encoded by using the generatedreference picture subjected to the predetermined picture processing.

An eleventh feature of the present invention is summarized as a movingpicture decoding device for decoding a moving picture constituted of atime sequence of frame pictures by motion compensation. The movingpicture decoding device includes a reference picture generation sectionconfigured to generate a plurality of different reference pictures byexecuting a plurality of different picture processing on a referenceframe picture; a decoding section configured to decode informationregarding a reference picture used for calculating a motion compensationvalue in a motion picture encoding device; and a motion compensationsection configured to calculate a motion compensation value for apredetermined area to be decoded by using the generated referencepicture specified by the information regarding the reference picture.The information regarding the reference picture is a combination ofidentification information of the reference frame picture andinformation indicating the picture processing.

A twelfth feature of the present invention is summarized as a movingpicture decoding device for decoding a moving picture constituted of atime sequence of frame pictures by motion compensation. The movingpicture decoding device includes a reference picture generation sectionconfigured to generate a reference picture subjected to predeterminedpicture processing from a reference frame picture in accordance with anencoding condition of a predetermined area to be decoded; and a motioncompensation section configured to calculate a motion compensation valuefor the predetermined area to be decoded by using the reference picturesubjected to the predetermined picture processing.

A thirteenth feature of the present invention is summarized as a movingpicture encoding method for encoding a moving picture constituted of atime sequence of frame pictures by motion compensation. The movingpicture encoding method includes a step A of generating a plurality ofdifferent reference pictures by executing a plurality of differentpicture processing on a reference frame picture; a step B of calculatinga motion compensation value for a predetermined area to be encoded byusing the generated reference picture; and a step C of transmitting acombination of information regarding the reference picture used forcalculating the motion compensation value and information indicating themotion compensation value. The information regarding the referencepicture is a combination of identification information of the referenceframe picture and information indicating the picture processing.

A fourteenth feature of the present invention is summarized as a movingpicture encoding method for encoding a moving picture constituted of atime sequence of frame pictures by motion compensation. The movingpicture encoding method includes a step A of generating a referencepicture subjected to predetermined picture processing from a referenceframe picture in accordance with an encoding condition of apredetermined area to be encoded; and a step B of calculating a motioncompensation value for the predetermined area to be encoded by using thegenerated reference picture subjected to the predetermined pictureprocessing.

A fifteenth feature of the present invention is summarized as a movingpicture decoding method for decoding a moving picture constituted of atime sequence of frame pictures by motion compensation. The movingpicture decoding method includes a step A of generating a plurality ofdifferent reference pictures by executing a plurality of differentpicture processing on a reference frame picture; a step B of decodinginformation regarding a reference picture used for calculating a motioncompensation value in a motion picture encoding device; and a step C ofcalculating a motion compensation value for a predetermined area to bedecoded by using the generated reference picture specified by theinformation regarding the reference picture. The information regardingthe reference picture is a combination of identification information ofthe reference frame picture and information indicating the pictureprocessing.

A sixteenth feature of the present invention is summarized as a movingpicture decoding method for decoding a moving picture constituted of atime sequence of frame pictures by motion compensation. The movingpicture decoding method includes a step A of generating a referencepicture subjected to predetermined picture processing from a referenceframe picture in accordance with an encoding condition of apredetermined area to be decoded; and a step B of calculating a motioncompensation value for the predetermined area to be decoded by using thegenerated reference picture subjected to the predetermined pictureprocessing.

A seventeenth feature of the present invention is summarized as aprogram for causing a computer to function as a moving picture encodingdevice for encoding a moving picture constituted of a time sequence offrame pictures by motion compensation. The moving picture encodingdevice includes a reference picture generation section configured togenerate a plurality of different reference pictures by executing aplurality of different picture processing on a reference frame picture;a motion compensation section configured to calculate a motioncompensation value for a predetermined area to be encoded by using thegenerated reference picture; and a transmission section configured totransmit a combination of information regarding the reference pictureused for calculating the motion compensation value and informationindicating the motion compensation value. The information regarding thereference picture is a combination of identification information of thereference frame picture and information indicating the pictureprocessing.

An eighteenth feature of the present invention is summarized as aprogram for causing a computer to function as a moving picture encodingdevice for encoding a moving picture constituted of a time sequence offrame pictures by motion compensation. The moving picture encodingdevice includes a reference picture generation section configured togenerate a reference picture subjected to predetermined pictureprocessing from a reference frame picture in accordance with encodingconditions of a predetermined area to be encoded; and a motioncompensation section configured to calculate a motion compensation valuefor the predetermined area to be encoded by using the generatedreference picture subjected to the predetermined picture processing.

A nineteenth feature of the present invention is summarized as a programfor causing a computer to function as a moving picture decoding devicefor decoding a moving picture constituted of a time sequence of framepictures by motion compensation. The moving picture decoding deviceincludes a reference picture generation section configured to generate aplurality of different reference pictures by executing a plurality ofdifferent picture processing on a reference frame picture; a decodingsection configured to decode information regarding a reference pictureused for calculating a motion compensation value in a motion pictureencoding device; and a motion compensation section configured tocalculate a motion compensation value for a predetermined area to bedecoded by using the generated reference picture specified by theinformation regarding the reference picture. The information regardingthe reference picture is a combination of identification information ofthe reference frame picture and information indicating the pictureprocessing.

A twentieth feature of the present invention is summarized as a programfor causing a computer to function as a moving picture decoding devicefor decoding a moving picture constituted of a time sequence of framepictures by motion compensation. The moving picture decoding deviceincludes a reference picture generation section configured to generate areference picture subjected to predetermined picture processing from areference frame picture in accordance with an encoding condition of apredetermined area to be decoded; and a motion compensation sectionconfigured to calculate a motion compensation value for thepredetermined area to be decoded by using the generated referencepicture subjected to the predetermined picture processing.

A twenty-first feature of the present invention is summarized as acomputer readable recording medium which stores a program for causing acomputer to function as a moving picture encoding device for encoding amoving picture constituted of a time sequence of frame pictures bymotion compensation. The moving picture encoding device includes areference picture generation section configured to generate a pluralityof different reference pictures by executing a plurality of differentpicture processing on a reference frame picture; a motion compensationsection configured to calculate a motion compensation value for apredetermined area to be encoded by using the generated referencepicture; and a transmission section configured to transmit a combinationof information regarding the reference picture used for calculating themotion compensation value and information indicating the motioncompensation value. The information regarding the reference picture is acombination of identification information of the reference frame pictureand information indicating the picture processing.

In the twenty-first feature of the invention, it is preferable that themotion compensation section is configured to switch the referencepicture used for calculating the motion compensation value by a unit fordetecting the motion vector, and that the transmission section isconfigured to transmit the combination of the information regarding thereference picture and the information indicating the motion compensationvalue by a unit for detecting the motion vector.

In the twenty-first feature of the invention, it is preferable that theinformation regarding the reference picture is a combination ofidentification information indicating a unit for detecting the motionvector and the information indicating the picture processing, and thatthe transmission section is configured to transmit a combination of theinformation regarding the reference picture, the identificationinformation of the reference frame picture and the informationindicating the motion compensation value for each predetermined area tobe encoded.

Additionally, in the twenty-first feature of the invention, it ispreferable that the picture processing is a processing of changing spaceresolution, and that the motion compensation section is configured toreduce accuracy of the motion vector used for calculating the motionvector when the reference picture of low space resolution is used.

Additionally, in the twenty-first feature of the invention, it ispreferable that, the information regarding the reference picturedynamically changes the combination of the identification information ofthe reference frame picture and the information indicating the pictureprocessing, in accordance with an encoding condition of thepredetermined area to be encoded.

Further, in the twenty-first feature of the invention, it is preferablethat the information regarding the reference picture dynamically changesthe combination of the identification information indicating the unitfor detecting the motion vector and the information indicating thepicture processing, in accordance with an encoding condition of thepredetermined area to be encoded.

A twenty-second feature of the present invention is a computer readablerecording medium which stores a program for causing a computer tofunction as a moving picture encoding device for encoding a movingpicture constituted of a time sequence of frame pictures by motioncompensation. The moving picture encoding device includes a referencepicture generation section configured to generate a reference picturesubjected to predetermined picture processing from a reference framepicture in accordance with an encoding condition of a predetermined areato be encoded; and a motion compensation section configured to calculatea motion compensation value for the predetermined area to be encoded byusing the generated reference picture subjected to the predeterminedpicture processing.

In the twenty-second feature of the invention, it is preferable that thereference picture generation section is configured to generate thereference picture subjected to the predetermined picture processing inaccordance with a type of a unit for detecting a motion vector.

Additionally, in the twenty-second feature of the invention, it ispreferable that the reference picture generation section is configuredto generate the reference picture subjected to the predetermined pictureprocessing in accordance with a quantization step.

A twenty-third feature of the present invention is a computer readablerecording medium which stores a program for causing a computer tofunction as a moving picture decoding device for decoding a movingpicture constituted of a time sequence of frame pictures by motioncompensation. The moving picture decoding device includes a referencepicture generation section configured to generate a plurality ofdifferent reference pictures by executing a plurality of differentpicture processing on a reference frame picture; a decoding sectionconfigured to decode information regarding a reference picture used forcalculating a motion compensation value in a motion picture encodingdevice; and a motion compensation section configured to calculate amotion compensation value for a predetermined area to be decoded byusing the generated reference picture specified by the informationregarding the reference picture. The information regarding the referencepicture is a combination of identification information of the referenceframe picture and information indicating the picture processing.

In the twenty-third feature of the invention, it is preferable that thedecoding section is configured to decode information regarding thereference picture and information indicating the motion compensationvalue by a unit for detecting a motion vector, and that the motioncompensation section is configured to switch the reference picture usedfor calculating the motion compensation value by the unit for detectingthe motion vector.

In the twenty-third feature of the invention, it is preferable that theinformation regarding the reference picture is a combination ofidentification information indicating a unit for detecting the motionvector and information indicating the picture processing, and that thedecoding section is configured to decode the information regarding thereference picture, identification information of the reference framepicture, and information indicating the motion compensation value by aunit of the predetermined area to be decoded. In addition, it ispreferable that the motion compensation section is configured tocalculate a motion compensation value for the predetermined area to bedecoded by using the generated reference picture specified by theinformation regarding the reference picture and the identificationinformation of the reference frame picture.

Additionally, in the twenty-third feature of the invention, it ispreferable that the picture processing is a processing of changing spaceresolution, and that the motion compensation section is configured toreduce accuracy of the motion vector used for calculating the motioncompensation value when the reference picture of low space resolution isused.

Additionally, in the twenty-third feature of the invention, it ispreferable that the information regarding the reference picturedynamically changes the combination of the identification information ofthe reference frame picture and the information indicating the pictureprocessing, in accordance with encoding conditions of the predeterminedarea to be decoded.

Further, in the twenty-third feature of the invention, it is preferablethat the information regarding the reference picture dynamically changesthe combination of the identification information indicating the unitfor detecting the motion vector and the information indicating thepicture processing, in accordance with an encoding condition of thepredetermined area to be decoded.

A twenty-fourth feature of the present invention is a computer readablerecording medium which stores a program for causing a computer tofunction as a moving picture decoding device for decoding a movingpicture constituted of a time sequence of frame pictures by motioncompensation. The moving picture decoding device includes a referencepicture generation section configured to generate a reference picturesubjected to predetermined picture processing from a reference framepicture in accordance with an encoding condition of a predetermined areato be decoded; and a motion compensation section configured to calculatea motion compensation value for the predetermined area to be decoded byusing the generated reference picture subjected to the predeterminedpicture processing.

Additionally, in the twenty-fourth feature of the invention, it ispreferable that the reference picture generation section is configuredto generate the reference picture subjected to the predetermined pictureprocessing, in accordance with a type of a unit for detecting a motionvector.

Further, in the twenty fourth-feature of the invention, it is preferablethat the reference picture generation section is configured to generatethe reference picture subjected to the predetermined picture processing,in accordance with a quantization step.

A twenty-fifth feature of the present invention is a moving pictureencoding device for encoding a moving picture constituted of a timesequence of frame pictures by motion compensation. The moving pictureencoding device includes a reference picture generation sectionconfigured to generate a plurality of different reference pictures byexecuting a plurality of different picture processing on a referenceframe picture; a 3-dimensional motion vector generation sectionconfigured to generate a 3-dimensional motion vector by correlating amotion vector detected by using the reference picture with informationindicating picture processing executed for the reference picture; amotion compensation section configured to calculate a motioncompensation value for a predetermined area to be encoded by using thegenerated reference picture; and a transmission section configured totransmit a combination of the 3-dimensional motion vector andinformation indicating the motion compensation value.

A twenty-sixth feature of the present invention is a moving picturedecoding device for decoding a moving picture constituted of a timesequence of frame pictures by motion compensation. The moving picturedecoding device includes a reference picture generation sectionconfigured to generate a plurality of different reference pictures byexecuting a plurality of different picture processing on a referenceframe picture; a decoding section configured to decode a 3-dimensionalmotion vector of a predetermined area to be decoded; and a motioncompensation section configured to calculate a motion compensation valuefor the predetermined area to be decoded by using the generatedreference picture specified by the 3-dimensional motion vector.

A twenty-seventh feature of the present invention is a moving pictureencoding method for encoding a moving picture constituted of a timesequence of frame pictures by motion compensation. The moving pictureencoding method includes a step A of generating a plurality of differentreference pictures by executing a plurality of different pictureprocessing on a reference frame picture; a step B of generating a3-dimensional motion vector by correlating a motion vector detected byusing the reference picture with information indicating pictureprocessing executed for the reference picture; a step C of calculating amotion compensation value for a predetermined area to be encoded byusing the generated reference picture; and a step D of transmitting acombination of the 3-dimensional motion vector and informationindicating the motion compensation value.

A twenty-eighth feature of the present invention is a moving picturedecoding method for decoding a moving picture constituted of a timesequence of frame pictures by motion compensation. The moving picturedecoding method includes a step A of generating a plurality of differentreference pictures by executing a plurality of different pictureprocessing on a reference frame picture; a step B of decoding a3-dimensional motion vector of a predetermined area to be decoded; and astep C of calculating a motion compensation value for the predeterminedarea to be decoded by using the generated reference picture specified bythe 3-dimensional motion vector.

A twenty-ninth feature of the present invention is a program for causinga computer to function as a moving picture encoding device for encodinga moving picture constituted of a time sequence of frame pictures bymotion compensation. The moving picture encoding device includes areference picture generation section configured to generate a pluralityof different reference pictures by executing a plurality of differentpicture processing on a reference frame picture; a 3-dimensional motionvector generation section configured to generate a 3-dimensional motionvector by correlating a motion vector detected by using the referencepicture with information indicating picture processing executed for thereference picture; a motion compensation section configured to calculatea motion compensation value for a predetermined area to be encoded byusing the generated reference picture; and a transmission sectionconfigured to transmit a combination of the 3-dimensional motion vectorand information indicating the motion compensation value.

A thirtieth feature of the present invention is a program for causing acomputer to function as a moving picture decoding device for decoding amoving picture constituted of a time sequence of frame pictures bymotion compensation. The moving picture decoding device includes areference picture generation section configured to generating aplurality of different reference pictures by executing a plurality ofdifferent picture processing on a reference frame picture; a decodingsection configured to decode a 3-dimensional motion vector of apredetermined area to be decoded; and a motion compensation sectionconfigured to calculate a motion compensation value for thepredetermined area to be decoded by using the generated referencepicture specified by the 3-dimensional motion vector.

A thirty-first feature of the present invention is a computer readablerecording medium which stores a program for causing a computer tofunction as a moving picture encoding device for encoding a movingpicture constituted of a time sequence of frame pictures by motioncompensation. The moving picture encoding device includes a referencepicture generation section configured to generate a plurality ofdifferent reference pictures by executing a plurality of differentpicture processing on a reference frame picture; a 3-dimensional motionvector generation section configured to generate a 3-dimensional motionvector by correlating a motion vector detected by using the referencepicture with information indicating picture processing executed for thereference picture; a motion compensation section configured to calculatea motion compensation value for a predetermined area to be encoded byusing the generated reference picture; and a transmission sectionconfigured to transmit a combination of the 3-dimensional motion vectorand information indicating the motion compensation value.

In the thirty-first feature of the invention, it is preferable that thereference picture generation section is configured to generate theplurality of different reference pictures by executing filter processingusing a filter which has a plurality of different pass bands, and thatthe 3-dimensional motion vector identifies the filter.

Additionally, in the thirty-first feature of the invention, it ispreferable that a 3-dimensional motion vector prediction section isprovided for predicting a 3-dimensional motion vector by using acorrelation between an encoded predetermined area in the frame pictureand the predetermined area to be encoded, and that the transmissionsection is configured to transmit a combination of differenceinformation between the 3-dimensional motion vector generated by the3-dimensional motion vector generation section and the 3-dimensionalmotion vector predicted by the 3-dimensional motion vector predictionsection as well as the information indicating the motion compensationvalue.

Additionally, in the thirty-first feature of the invention, it ispreferable that the 3-dimensional motion vector prediction section isconfigured to predict the 3-dimensional motion vector by switching acontext in arithmetic encoding.

Further, in the thirty-first feature of the invention, it is preferablethat the picture processing is a processing of changing spaceresolution, and that the 3-dimensional motion vector generation sectionis configured to reduce accuracy of a 3-dimensional motion vector for areference picture of low space resolution.

A thirty-second feature of the present invention is a computer readablerecording medium which stores a program for causing a computer tofunction as a moving picture decoding device for decoding a movingpicture constituted of a time sequence of frame pictures by motioncompensation. The moving picture decoding device includes a referencepicture generation section configured to generate a plurality ofdifferent reference pictures by executing a plurality of differentpicture processing on a reference frame picture; a decoding sectionconfigured to decode a 3-dimensional motion vector of a predeterminedarea to be decoded; and a motion compensation section configured tocalculate a motion compensation value for the predetermined area to bedecoded by using the generated reference picture specified by the3-dimensional motion vector.

In the thirty-second feature of the invention, it is preferable that thereference picture generation section is configured to generate theplurality of different reference pictures by executing filter processingusing a filter which has a plurality of different pass bands, and thatthe 3-dimensional motion vector identifies the filter.

Additionally, in the thirty-second feature of the invention, it ispreferable that a 3-dimensional motion vector prediction section isprovided for predicting a 3-dimensional motion vector by using acorrelation between a decoded predetermined area in the frame pictureand the predetermined area to be decoded, and that the motioncompensation section is configured to calculate a motion compensationvalue for the predetermined area to be decoded by using differenceinformation between the 3-dimensional motion vector decoded by thedecoding section and the 3-dimensional motion vector predicted by the3-dimensional motion vector prediction section.

Furthermore, in the thirty-second feature of the invention, it ispreferable that the 3-dimensional motion vector prediction section isconfigured to predict the 3-dimensional motion vector by switching acontext in arithmetic encoding.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram of a moving picture encoding deviceaccording to a conventional technology.

FIG. 2 is a schematic diagram of a moving picture decoding deviceaccording to the conventional technology.

FIG. 3 is a diagram showing divided patterns of a macro-block in anINTER prediction mode according to the conventional technology.

FIG. 4 is a diagram showing a concept of a funny position according tothe conventional technology.

FIG. 5 is a diagram showing a method for calculating a predicted motionvector in a moving picture encoding device according to an embodiment ofthe present invention.

FIG. 6 is a functional block diagram of a motion compensation section ofthe moving picture encoding device according to the embodiment of theinvention.

FIG. 7 is a diagram showing a concept for determining a “funny position”in the moving picture encoding device according to the embodiment of theinvention.

FIG. 8 is a flowchart showing an operation of the motion compensationsection of the moving picture encoding device according to theembodiment of the invention.

FIG. 9 is a flowchart showing a decoding process in the moving picturedecoding device according to the embodiment of the present invention.

FIG. 10 is a diagram showing a concept for determining a “funnyposition” in a moving picture encoding device according to a modifiedexample of the present invention.

FIG. 11 is a diagram showing a concept for determining a “funnyposition” in the moving picture encoding device according to themodified example of the invention.

FIG. 12 is a schematic diagram of a moving picture encoding deviceaccording to an embodiment of the present invention.

FIG. 13 is a schematic diagram of a moving picture decoding deviceaccording to an embodiment of the present invention.

FIG. 14 is a diagram showing an encoding syntax in a macro-block unit inan H. 26L encoding system used in the embodiment of the invention.

FIG. 15 is an example of a reference frame code table used in theembodiment of the invention.

FIG. 16 is a part of the reference frame code table used in theembodiment of the invention.

FIG. 17 is a diagram showing an encoding syntax in a macro-block unit inan H. 26L encoding system used in a modified example of the presentinvention.

FIG. 18 is a table of macro-block mode codes used in the modifiedexample of the invention.

FIG. 19 is a schematic diagram of a moving picture encoding deviceaccording to an embodiment of the present invention.

FIG. 20 is a schematic diagram of a moving picture decoding deviceaccording to the embodiment of the present invention.

FIG. 21 is a diagram explaining hierarchical reference picturesaccording to the embodiment of the invention.

FIG. 22 is a diagram explaining a method for generating the hierarchicalreference pictures according to the embodiment of the invention.

FIG. 23 is a diagram showing a method for calculating a predicted motionvector in the moving picture encoding device according to the embodimentof the invention.

FIG. 24 is a flowchart showing a motion compensation operation in themoving picture encoding device according to the embodiment of theinvention.

FIG. 25 is a diagram explaining a method for generating hierarchicalreference pictures according to a modified example of the presentinvention.

FIG. 26 is a diagram showing a computer readable recording medium whichstores a program for causing a computer to function as the movingpicture encoding device or the moving picture decoding device of theembodiment of the invention.

BEST EMBODIMENT OF THE INVENTION

(Embodiment 1)

In the first embodiment of the present invention, description will bemade of a moving picture encoding device 20 and a moving picturedecoding device 50 in which improvements are introduced in motioncompensation in a “funny position” (a first problem) which has been aproblem with an “H. 26L encoding system” defined by a conventional“TML-8”.

According to the embodiment, except for motion compensation in the“funny position”, operations thereof are similar to those of the movingpicture encoding device 20 and the moving picture decoding device 50described in the “TML-8”. Thus, details thereof are omitted, anddescription will focus on the differences.

Specifically, the difference in configuration of the moving pictureencoding device 20 and the moving picture decoding device 50 between theembodiment of the present invention and the conventional embodiment liesin the difference in configuration of the motion compensation sections33 and 72 between the present and conventional technology.

In the embodiment of the present invention, since the motioncompensation section 33 of the moving picture encoding device 20 and themotion compensation section 72 of the moving picture decoding device 50have the same configuration. Thus, the motion compensation section 33 ofthe moving picture encoding device 20 will be described hereinafter.

Incidentally, the moving picture encoding device 20 of the embodiment isconfigured to encode a moving picture (an input video signal 1)constituted of a time sequence of frame pictures by motion compensation.The moving picture decoding device 50 of the embodiment is configured todecode a moving picture (an output video signal 1A) constituted of atime sequence of frame pictures by motion compensation.

Additionally, in the moving picture encoding device 20 of theembodiment, a motion detection section 32 constitutes a motion vectordetection section configured to detect a motion vector 3 of apredetermined area (e.g., a macro-block) to be encoded in the framepicture. In the moving picture decoding device 50 of the embodiment, avariable length decoding section 71 constitutes a motion vector decodingsection configured to decode a motion vector 3 of a predetermined area(e.g., a macro-block) to be decoded in the frame picture.

As shown in FIG. 6, the motion compensation section 33 of the movingpicture encoding device 20 of the embodiment includes a motion vectorinput section 33 a, a reference frame picture input section 33 b, apredicted motion vector calculation section 33 c, a determinationsection 33 d, and a predicted picture signal generation section 33 e.

According to the embodiment, the predicted motion vector calculationsection 33 c constitutes a prediction section configured to predict apredicted motion vector PMV_(E)=(PMVx_(E), PMVy_(E)) of a predeterminedarea (a macro-block E) to be encoded, by using an encoded motion vector(e.g., MV_(A)=(MVx_(A), MVy_(A)), MV_(B)=(MVx_(B), MVy_(B)),MV_(C)=(MVx_(C), MVy_(C)) of a predetermined area in the frame picture.

Additionally, the determination section 33 d constitutes a determinationsection configured to determine whether or not a motion vectorMV_(E)=(MVx_(E), MVy_(E)) detected by the motion vector detectionsection (the motion detection section 32) is a predetermined motionvector (the motion vector indicating a “funny position”) set inaccordance with a motion vector MPV_(E=(PMVx) _(E), PMVy_(E)) predictedby the prediction section (the predicted motion vector calculationsection 33 c).

Further, the predicted picture signal generation section 33 econstitutes a switching section configured to switch a method forcalculating a “motion compensation value” of the predetermined area tobe encoded (the generation method of predicted picture signal 6),depending on whether or not the motion vector MV_(E)=(MVx_(E), MVy_(E))detected by the motion vector detection section (the motion detectionsection 32) is a predetermined motion vector (the motion vectorindicating a “funny position”).

The motion vector input section 33 a is connected to the determinationsection 33 d, and is configured to receive the motion vector MV_(E)detected by the motion detection section 32, so as to transmit themotion vector MV_(E) to the determination section 33 d.

The reference frame picture input section 33 b is connected to thepredicted motion vector calculation section 33 c and the determinationsection 33 d, and is configured to extract motion vectors of neighboringareas (macro-blocks A, B and C) of the predetermined area to be encoded,which are stored in a frame memory 34, so as to transmit the vectors tothe predicted motion vector calculation section 33 c. Also, thereference frame picture input section 33 is configured to extract areference frame picture 5 stored in the frame memory 34, so as totransmit the reference frame picture 5 to the determination section 33d.

The predicted motion vector calculation section 33 c is connected to thereference frame picture input section 33 b and the determination section33 d, and is configured to calculate a predicted motion vectorPMV_(E)=(PMVx_(E), PMVy_(E)) which is a predicted value of a motionvector MV_(E)=(MVx_(E), MVy_(E)) of the predetermined area (themacro-block) to be encoded, by using, e.g., the motion vectors (encodedmotion vectors of the predetermined areas in the frame picture) MV_(A),MV_(B) and MV_(C) of the neighboring areas (the macro-blocks A, B and C)of the predetermined area to be encoded, which are stored in the framememory 34.

Here, PMVx_(E) indicates a horizontal element (an X element) of apredicted motion vector, and PMVy_(E) indicates a vertical element (a Yelement) of the predicted motion vector.

According to the “TML-8”, in order to efficiently encode the motionvector, a motion vector of a predetermined area to be encoded ispredicted and encoded by a prediction system called “median prediction”through using an encoded motion vector of a neighboring area included inthe reference frame picture 5.

In FIG. 5, since MV_(A)=(MVx_(A), MVy_(A)), MV_(B)=(MVx_(B), MVy_(B))and MV_(C)=(MVx_(C), MVy_(C)) which are motion vectors MV of theneighboring areas (macro-blocks) A, B and C have been encoded, anaverage of the horizontal elements MVx_(A), MVx_(B) and MVx_(C) of themotion vectors is obtained to be set as the horizontal element PMVx_(E)for a predicted motion vector of the predetermined area (themacro-block) E to be encoded, and an average of the vertical elementsMVy_(A), MVy_(B) and MVy_(C) of the motion vectors is obtained to be setas a vertical element PMVy_(E) of a predicted motion vector of thepredetermined area (the macro-block) E to be encoded.

For example, in the case of a prediction mode (an encoding mode) inwhich the neighboring areas (the macro-blocks) A, B and C used forcalculating the predicted motion vector PMV are outside the framepicture or have no motion vectors, a predicted motion vector PMV is setas a zero vector.

Further, when 3 or more motion vectors are not present in theneighboring areas (the macro-blocks) A, B and C used for calculating thepredicted motion vector PMV of the predetermined area (the macro-block)E to be encoded, a value of a predicted motion vector PMV_(E) of thepredetermined area (the macro-block) E to be encoded is always obtained,by using an assumption that the motion vectors of the neighboring areas(the macro-blocks) A, B and C are set to zero vectors or otherassumptions.

Additionally, the predicted motion vector calculation section 33 ccalculates difference information MVD=(MVDx, MVDy) between the motionvector MV_(E) from the motion vector input section 33 a and thepredicted motion vector PMV_(E). Here, MVDx which is an X element of thedifference information is calculated by “MVx_(E)−PMVx_(E)”, and MVDywhich is a Y element of the difference information MVD is calculated by“MVy_(E)-PMVy_(E)”.

In the “H. 26L encoding system”, in order to improve transmissionefficiency, the motion vector MV is encoded in a form of theaforementioned difference information MVD so as to be transmitted.

The determination section 33 d is connected to the motion vector inputsection 33 a, the reference frame picture input section 33 b, thepredicted motion vector calculation section 33 c and the predictedpicture signal generation section 33 e. The determination section 33 dis configured to determine a method of generating a predicted picturesignal 6 (a method for calculating a motion compensation value), so asto put the method to the predicted picture signal generation section 33e, in accordance with the motion vector MV_(E) from the motion vectorinput section 33 a and the predicted motion vector PMV_(E) from thepredicted motion vector calculation section 33 c.

Specifically, as shown in FIG. 7, the determination section 33 ddetermines whether or not the motion vector MV_(E) indicates a “funnyposition” in accordance with a phase of the vertical component PMVy_(E)of the predicted motion vector generated by the predicted picture signalgeneration section 33 e and the motion vector MV_(E) detected by themotion detection section 32, and puts a result of the determination tothe predicted picture signal generation section 33 e.

Hereinafter, a unit regarding expression of the predicted motion vectorPMV_(E) and the motion vector MV_(E) is a ¼ pixel.

First, in the case of “PMVy_(E)%4=0 or 1” (i.e., PMVy_(E) is a firstphase”), “MVx_(E)%4=3” and “MVy_(E)%4=3”, the determination section 33 ddetermines that the motion vector MV_(E) (MVx_(E), MVy_(E)) of thepredetermined area (the macro-block E) to be encoded indicates a “funnyposition”.

Second, in the case of “PMVy_(E)%4=2 or 3” (i.e., PMVy_(E) is a secondphase), “MVx_(E)%4=1” and “MVy_(E)%4=1”, the determination section 33 ddetermines that the motion vector MV_(E) (MVx_(E), MVy_(E)) of thepredetermined area (the macro-block E) to be encoded indicates the“funny position.”

As a result, an area indicated by the predicted motion vector PMV_(E) isadjusted so as not to be superimposed on the “funny position”. That is,the predicted motion vector PMV_(E) predicted by the predicted motionvector calculation section 33 c (the prediction section) is set to bedifferent from the motion vector (the predetermined motion vector)indicating the “funny position”.

In the case that PMVy_(E) is a first phase, the motion compensation ofthe conventional “H. 26L encoding system” is applied as it is (i.e., themotion vector indicating a pixel position a (see FIG. 7) indicates the“funny position”).

Additionally, in the case that PMVy_(E) is a second phase, a motionvector indicating a different pixel position b (see FIG. 7) is supposedto indicate the “funny position”. However, an obtained “motioncompensation value” is smoothed as an average of pixel values of 4neighboring integer pixel positions which surround the motion vector,and other motion compensation processing is similar to the motioncompensation processing of the conventional “H. 26L encoding system”.

The predicted picture signal generation section 33 e is connected to thedetermination section 33 d, and is configured to generate a predictedpicture signal 6, by switching the “generation method of a predictedpicture signal 6” regarding the predetermined area (the macro-block) tobe encoded.

Specifically, when it is determined that the motion vector MV_(E) of thepredetermined area (the macro-block E) to be encoded, which is detectedby the motion detection section 32, indicates the “funny position”, thepredicted picture signal generation section 33 e smoothes the predictedpicture signal 6 of the predetermined area as an average of pixel valuesof the 4 neighboring integer pixel positions which surround the pixelposition indicated by the motion vector MV_(E), and generates thepredicted picture signal 6 of the predetermined area by the conventional“H. 26L encoding system” in other cases.

Incidentally, in the present embodiment, the motion compensation section33 and the motion detection section 32 are separately disposed. However,the motion compensation section 33 and the motion detection section 32may be integrally disposed.

FIG. 8 shows operation of the aforementioned motion compensation section33.

In step 401, the predicted motion vector calculation section 33 ccalculates a predicted motion vector PMV_(E) which is a predicted valueof a motion vector of a predetermined area (a macro-block E in FIG. 5)to be encoded in a frame picture, based on encoded motion vectorsMV_(A), MV_(B) and MV_(C) of neighboring areas (the macro-blocks A, Band C in FIG. 5) in the same frame picture.

In step 402, the determination section 33 d determines whether or not amotion vector MV_(E) indicates a “funny position”, in accordance with aphase of a vertical element PMVy_(E) of the predicted motion vector fromthe predicted motion vector calculation section 33 c and the motionvector MV_(E) from the motion vector input section 33 a.

In step 403, when the aforementioned motion vector MV_(E) indicates the“funny position”, the predicted picture signal generation section 33 egenerates a predicted picture signal 6 of the predetermined area in asmoothed form.

In step 404, when the aforementioned motion vector MV_(E) does notindicate the “funny position”, the predicted picture signal generationsection 33 e generates a predicted picture signal 6 of the predeterminedarea by the conventional “H. 26L encoding system”.

Next, steps for a decoding process in the moving picture decoding device50 will be described with reference to FIG. 9.

In step 501, the variable length decoding section 71 detects a“synchronous word” which indicates a head of a picture (each framepicture constituting an input video signal 1).

In step 502, the variable length decoding section 71 decodes a “pictureheader” of the aforementioned picture. The “picture header” contains“picture type information” for determining whether the picture is a“picture which encodes all the macro-blocks constituting the picture byan INTRA prediction mode (hereinafter referred to as “I picture”)” or a“picture which uses an INTER prediction mode (hereinafter referred to asa “P picture”).” Also, the picture header contains a value of aquantization parameter in an orthogonal transformation coefficient andthe like.

Subsequently, the process proceeds to the decoding of data of eachmacro-block layer constituted of a predetermined syntax.

In step 503, the variable length decoding section 71 decodes a “RUN” inthe macro-block layer. The “RUN” indicates the number of repeatedmacro-blocks in which data of the macro-block layer is zero, andmacro-blocks (skip MBs) to which as many skip modes as the number of the“RUN” are applied are generated.

In step 504, it is determined whether or not a macro-block to be decodedis a skip MB.

If the macro-block is a skip MB, in step 505, an area of 16×16 pixels inthe same position on a predetermined reference frame picture 5 stored inthe frame memory 73 is employed, as it is, for a predicted picturesignal 6. This processing is carried out by transmitting a motion vectorwhose value is zero and an identification number of the predeterminedreference frame picture to the motion compensation section 72 by thevariable length decoding section 71.

If the macro-block is not a skip MB, in step 506, it is determinedwhether or not the “RUN” of the MB indicates the last MB of the picture.

If the “RUN” of the MB is the last MB, in step 507, the variable lengthdecoding of the picture is terminated, and a variable length decoding ofa next picture begins.

If the “RUN” is neither the skip MB nor the last MB, i.e., if the “RUN”is a normal MB, in step 508, the variable length decoding section 71decodes a “MB_Type (a macro-block type). By the “MB_Type”, a predictionmode 4 of the predetermined area (the macro-block) to be decoded isestablished. In step 509, it is determined whether or not theestablished prediction mode 4 is an “INTER prediction mode”.

If the prediction mode 4 is the “INTRA prediction mode”, in step 510,the variable length decoding section 71 decodes an “intra_pred_mode”. Instep 511, the space prediction section 74 executes space prediction froma pixel value of a neighboring area based on the “intra_pred_mode”, soas to generate a predicted picture signal 7.

If the prediction mode 4 is the “INTER prediction mode”, the predictionmode 4 is one of the modes 1 to 7 shown in FIG. 3. Thus, at this time,the numbers of “Ref_frames (reference frame picture numbers)” and “MVDs(difference information of motion vector)” to be decoded areestablished. In accordance with such information, the variable lengthdecoding section 71 decodes a combination of the “Ref_frame” and the“MVD”.

However, because determination of whether or not the “Ref_frame” hasbeen multiplexed is integrated into the aforementioned “picture typeinformation”, in step 512, it is determined whether or not the“Ref_frame” is present in accordance with a value of the “picture typeinformation”.

If the “Ref_frame” is present, in step 513, the variable length decodingsection 71 decodes the “Ref_frame”, and then, in step 514, the variablelength decoding section 71 decodes the “MVD”. If the “Ref_frame” is notpresent, only the “MVD” is decoded in step 514.

In step 514, based on the prediction mode 4 established by the“Ref_frame”, the “MVD” and the “MB_Type” which have been obtained,motion vectors MV corresponding to all the 4×4 blocks in the MB arerestored.

In step 515, the motion compensation section 72 generates a predictedpicture signal 6 for each of 4×4 blocks based on the “Rev_frame” and themotion vector MV. Processing regarding the “funny position” is reflectedhere.

In step 516, the variable length decoding section 71 restores aquantized orthogonal transformation coefficient 11. In step 517, theinverse quantization section 76 restores the orthogonal transformationcoefficient 10. In step 518, the inverse orthogonal transformationsection 77 restores a predicted residual signal 9.

In step 519, at the adder 78, a predicted picture signal 8 from theswitch 75 and the predicted residual signal 9 from the inverseorthogonal transformation section 77 are summed up to obtain a framepicture signal 2 of the MB. Then, the process proceeds to the decodingof a next MB.

Operations/Effects of Moving Picture Encoding and Decoding DevicesAccording to the Embodiment 1

According to the moving picture encoding device of the embodiment, thepredicted picture signal generation section 33 e switches the method ofcalculating a motion compensation value for the predetermined area to beencoded, in accordance with the determination result of thedetermination section 33 d, i.e., the predicted motion vector predictedby the predicted motion vector calculation section 33 c. For thisreason, it is possible to express a pixel value of the same pixelposition (N+¾ pixel, M+¾ pixel: N and M are given integers) as the“funny position”.

Moreover, it is possible to solve the problem of strong smoothing beingalways applied to the “motion compensation value” in the area (e.g., themacro-block or sub-block) having a motion vector which indicates thesame pixel position as the “funny position”.

The method of generating a predicted picture signal or a motioncompensation value described in the embodiment is merely an example. Agiven generation method necessary for realizing the switching of thecalculation method of the motion compensation value executed accordingto the embodiment can be used.

MODIFIED EXAMPLE 1A

Description will be made of a modified example 1A of the moving pictureencoding device 20 and the moving picture decoding device 50 of theforegoing Embodiment 1. Hereinafter, only differences from theEmbodiment 1 will be described.

With regard to the moving picture encoding device 20 and the movingpicture decoding device 50 according to the modified example,modification is introduced in the motion compensation section 33 of themoving picture encoding device 20 and the motion compensation section 72of the moving picture decoding device 50 of the foregoing embodiment.The motion compensation section 33 of the moving picture encoding device20 and the motion compensation section 72 of the moving picture decodingsection 50 are identical. Thus, hereinafter, the motion compensationsection 33 of the moving picture encoding device 20 will be described.

According to the modified example, the determination section 33 d of themotion compensation section 33 determines whether or not the motionvector indicates a “funny position” in accordance with a phase of ahorizontal element PMVx_(E) of a predicted motion vector and the motionvector MV_(E)=(MVx_(E), MVy_(E)) detected by the motion detectionsection 32. Subsequently, a result of the determination is reported tothe predicted picture signal generation section 33 e by thedetermination section 33 d. Hereinafter, a unit regarding expression ofthe motion vector MV_(E)=(MVx_(E), MVy_(E)) is a ¼ pixel.

First, in the case of “PMVx_(E)%4=0 or 1” (i.e., PMVx_(E) is a firstphase), “MVx_(E)%4=3” and “MVy_(E)%4=3”, the determination section 33 ddetermines that the motion vector MV_(E) (MVx_(E), MVy_(E)) indicates a“funny position”.

Second, in the case of “PMVx_(E)%4=2 or 3” (i.e., PMVx_(E) is a secondphase), “MVx_(E)%4=1” and “MVy_(E)%4=1,” the determination section 33 ddetermines that the motion vector MV_(E) (MVx_(E), MVy_(E)) indicatesthe “funny position”.

As a result, an area indicated by the predicted motion vector PMV_(E) isadjusted so as not to be superimposed on the “funny position”. That is,the motion vector indicating the “funny position” is set to be differentfrom the predicted motion vector PMV_(E).

As described above, the modified example 1A has an effect of reducing apossibility of superimposition of the motion vector which becomes a“funny position” on real motion, by using the predicted encodingstructure of the motion vector. In FIG. 5, for example, even if theblocks A, B, C, D and E move right downward in parallel by (¾ pixel, ¾pixel), i.e., even in the case of a motion vector MV=(MVx, MVy)=(¾, ¾),the motion vector MV=(MVx, MVy)=(¾, ¾) does not necessarily indicate the“funny position”, when the second phase of the predicted motion vectorPMV is “2”.

MODIFIED EXAMPLE 1B

Description will be made of a modified example 1B of the moving pictureencoding device 20 and the moving picture decoding device 50 of theforegoing Embodiment 1. Hereinafter, only differences from theEmbodiment 1 will be described.

In the moving picture encoding device 20 and the moving picture decodingdevice 50 according to the modified example, the motion compensationsection 33 of the moving picture encoding device 20 and the motioncompensation section 72 of the moving picture decoding device 50 of theforegoing embodiment are changed. The motion compensation section 33 ofthe moving picture encoding device 20 and the motion compensationsection 72 of the moving picture decoding section 50 are identical.Thus, hereinafter, the motion compensation section 33 of the movingpicture encoding device 20 will be described.

According to the modified example, the determination section 33 d of themotion compensation section 33 determines whether or not a motion vectorMV_(E) of a predetermined area (a macro-block E) to be encoded, which isdetected by the motion detection section 32, indicates a “funnyposition” in accordance with the aforementioned difference informationMVD_(E)=(MVDx_(E), MVDy_(E)). Subsequently, a result of thedetermination is reported to the predicted picture signal generationsection 33 e by the determination section 33 d.

That is, the determination section 33 d constitutes a determinationsection configured to determine that the motion vector MV_(E) detectedby the motion vector detection section (the motion detection section 32)is a predetermined motion vector (a motion vector indicating the “funnyposition”), when difference information MVD_(E) between a motion vectorPMV_(E) predicted by the prediction section (the predicted motion vectorcalculation section 33 c) and the motion vector MV_(E) detected by themotion vector detection section (the motion detection section 32) isequal to a predetermined value.

Hereinafter, a unit regarding expression of the motion vectorMV_(E)=(MVx_(E), MVy_(E)) is a ¼ pixel.

For example, in the case of “MVDx_(E)%4=3” and “MVDy_(E)%4=3”, thedetermination section 33 d determines that the motion vector MV_(E)(MVx_(E), MVy_(E)) detected by the motion detection section 32 indicatesa “funny position”.

In such a case, i.e., if a smoothing operation is carried out with aquotient remainder of the difference information MVD_(E) of the motionvector, it is necessary to change a method of calculating a pixel valuein the “funny position”.

That is, according to the conventional technology, the foregoingEmbodiment 1 and the Modified Example 1, the motion compensation iscarried out based on the average of the pixel values of the integerpixel positions which surround the “funny position”. However, ifdetermination of the “funny position” is carried out with the quotientremainder of the difference information MVD_(E) of the motion vector,the “funny position” itself is represented as an integer pixel position,and an integer pixel position for obtaining an average may not beestablished. Thus, a motion compensation operation is carried out asfollows.

In FIG. 11, because of “MVDx_(E)%4=3” and “MVDy_(E)%4=3” (in this case,“PMVx_(E)%4=1” and “PMVy_(E)%4=1”), the predicted picture generationsection 33 e determines that the motion vector MV_(E)=(MVx_(E), MVy_(E))detected by the motion detection section 32 indicates the “funnyposition”. Here, in reality, the motion vector MV_(E)=(MVx_(E), MVy_(E))indicates an integer pixel position “D”.

In such a case, as shown in FIG. 11, the predicted picture generationsection 33 e causes a motion compensation value of a predetermined area(a macro-block or sub-block) to take on an average of a pixel value of(MVx_(E)+2, MVy_(E)+2), a pixel value of (MVx_(E)+2, MVy_(E)−2), a pixelvalue of (MVx_(E)−2, MVy_(E)+2) and a pixel value of (MVx_(E)−2,MVy_(E)−2) in the reference frame picture 5.

Instead, the predicted picture generation section 33 e may obtain amotion compensation value of the predetermined area (the macro-block orsub-block) to be encoded, exclusively from a pixel value of an integerpixel position in the reference frame picture 5. Specifically, themotion compensation value is caused to take on an average of a pixelvalue of ((MVx_(E)/4)×4, (MVy_(E)/4)×4), a pixel value of(((MVx_(E)+4)/4)×4, (MVy_(E)/4)×4), a pixel value of (((MVx_(E)/4)×4,((MVy_(E)+4)/4)×4) and a pixel value of (((MVx_(E)+4)/4)×4,((MVy_(E)+4)/4)×4).

According to the Modified Example 1B, in the moving picture encodingdevice 20, choice can be made between transmission of a real motionvector (MVx_(E), MVy_(E)) and transmission of a motion vectorMV_(E)=(MVx_(E), MVy_(E))=(1, 1) for the purpose of executing smoothedmotion compensation.

(Embodiment 2)

Description will be made of a moving picture encoding device 20 and amoving picture decoding device 50 according to the Embodiment 2 of thepresent invention. In the embodiment, no motion compensation which usesa “funny position” is carried out.

The description for the Embodiment 1 has been referred tocountermeasures against the problem regarding the transmission of themotion vector MV (or difference information MVD of the motion vector)which indicates the “funny position”, i.e., the problem of theobstructed transmission of a real motion vector MV. However, there stillremains a possibility that the real motion vector MV can not betransmitted.

Thus, in the present embodiment, description will be made of the movingpicture encoding device 20 and the moving picture decoding device 50.The moving picture encoding and decoding devices can carry out motioncompensation from reference pictures whose degrees of smoothness aredifferent from one predetermined area to be encoded to another, byseparately preparing a predicted picture signal which is stronglysmoothed like provided by the “motion compensation value” in the “funnyposition” of the Embodiment 1 and a predicted picture signal of a normal“motion compensation value” in a position other than the “funnyposition”, and by signaling identification information of these 2 kindsof predicted picture signals together with a reference frame picturenumber. Thus, the motion compensation can be carried out without usingthe “funny position”.

FIG. 12 is a schematic view of the moving picture encoding device 20 ofthe embodiment, and FIG. 13 is a schematic view of the moving picturedecoding device 50.

In this embodiment, as in the case of the Embodiment 1, description willbe made of the moving picture encoding device 20 and the moving picturedecoding device 50 for which improvements have been introduced in motioncompensation in a “funny position” which is a problem in the “H. 26Lencoding system” defined by the conventional “TML-8” (a first problem).

In the Embodiment 2, operations are similar to those of the movingpicture encoding device 20 and the moving picture decoding device 50described in the “TML-8”, except that the motion compensation isexecuted from a different reference picture without using “funnypositions”. Thus, details thereof are omitted, and description willfocus on differences.

Basic operations of the moving picture encoding device 20 and the movingpicture decoding device 50 according to the present embodiment arevirtually the same as those of the moving picture encoding device 20 andthe moving picture decoding device 50 according to the conventionaltechnology, except that modifications have been introduced in theconfiguration of the motion compensation sections 33 and 72 as well asthe variable encoding section 40, and except that new reference picturegeneration sections 45 and 80 have been added.

According to the present embodiment, the reference picture generationsections 45 and 80 are configured to generate a plurality of differentreference pictures (normal first reference pictures or second referencepictures of strong smoothing application) 17, by executing a pluralityof different picture processing on a reference frame picture 5. Here, asthe aforementioned picture processing, processing for changing degreesof smoothness, processing for changing degrees of space resolution, andthe like are conceived. In the present embodiment, as pictureprocessing, a case of using processing for changing a degree ofsmoothness will be described.

Additionally, the motion compensation section 33 is configured tocalculate a motion compensation value (a predicted picture signal 6) fora predetermined area (a macro-block) to be encoded, by using thereference picture 17 in place of the reference frame picture 5.

The variable length encoding section 40 constitutes a transmissionsection configured to transmit a combination of “information (areference frame code: Ref_frame) regarding the reference picture 17”used for calculating the motion compensation value (the predictedpicture signal 6) and “information (a predicted residual signal dataencoding syntax: Texture Coding Syntax) indicating the motioncompensation value”.

Additionally, the variable length decoding section 71 constitutes adecoding section configured to decode the “information (the Ref_frame)regarding the reference picture” used for calculating the motioncompensation value in the moving picture encoding device 20, and the“information (the predicted residual signal data encoding syntax)indicating the motion compensation value”.

The variable length decoding section 71 transmits a motion vector 3, aprediction mode 4, and a “reference frame code (an Ref_frame)” 4A, tothe motion compensation 72.

Further, the motion compensation section 72 is configured to calculate amotion compensation value for a predetermined area (a macro-block) to beencoded, by using the reference picture 17 specified by the “information(the Ref_frame) regarding the reference picture” in place of thereference frame picture 5.

According to the present embodiment, the “information (the referenceframe code: Ref_frame) regarding the reference picture 17” is acombination of “identification information (a reference frame picturenumber) of the reference frame picture” and “information (a firstreference picture or second reference picture) indicating a degree ofsmoothness”.

First, according to the present embodiment, the reference picturegeneration section 45 generates a normal reference picture (hereinafter,referred to as a first reference picture) which has no “funny position”.A “motion compensation value” of the first reference picture is equal toa “motion compensation value” in each pixel position of an integer pixelposition, a ½ pixel position and a ¼ pixel position of the “TML-8”,except that an original “motion compensation value” is used even for a“motion compensation value” in the same position as the “funny position”of the TML-8”.

The “motion compensation value” of the first reference picture in thesame pixel position as the “funny position” of the “TML-8” is generatedas an average of four points of pixel values of a neighboring integerpixel position and a neighboring ½ pixel position as in the case of the“motion compensation value” in a position (¼ pixel, ¼ pixel) other thanthe “funny position” of the “TML-8”.

Second, a reference picture (hereinafter, referred to as a secondreference picture) which is strongly smoothed like provided by the“motion compensation value” in the “funny position” of the “TML-8” isgenerated by using the first reference picture. Here, the secondreference picture can be generated by executing picture processing withvarious kinds of smoothing filers for each pixel value of the firstreference picture. For example, the second reference picture having ¼pixel accuracy can be generated by independently operating 3 tap filters(1, 4, 1)/6 having smoothing effects vertically and horizontally for thepixel value of each pixel position of the first reference picture having¼ pixel accuracy.

According to the “H. 26L encoding system”, a plurality of encoded framepictures which differ from one another with time are prepared asreference frame pictures 5, and these can be used as reference picturesfor motion compensation. Additionally, bits of identificationinformation of these encoded frame pictures which differ from oneanother with time are discriminated as reference frame picture numbers.

According to the embodiment, identification information of the firstreference picture or identification information of the second referencepicture, i.e., information of a method of generating a reference pictureand the reference frame picture number are combined to be transmitted.

Thus, without using the “funny position”, it is possible to carry outmotion compensation using reference pictures whose degrees of smoothnessare different from one predetermined area (e.g., a macro-block) to beencoded to another.

In this case, for the reference frame pictures 5 which are encoded framepictures different from one anointer with time, the reference picturegeneration sections 45 and 80 generate a first and a second referencepictures whose degrees of smoothness are different. Consequently, thesereference frame pictures can be used as “reference pictures” for motioncompensation in the motion compensation sections 33 and 72.

FIG. 14 shows an “encoding syntax for each macro-block unit according tothe H. 26L encoding system” used in the embodiment. According to theembodiment, there is no variation from the encoding syntax for eachmacro-block unit of the H. 26L encoding system. However, definition ofan “Ref_frame” is changed to a combination of a “reference frame picturenumber” and “identification information of a method of generating areference picture”, i.e., a “reference frame code”.

As shown in FIG. 14, even when a prediction mode (e.g., a mode 7) whichrequires a plurality of motion vectors to be detected in one macro-blockis applied, encoding can be carried out without containing informationregarding a plurality of “MB_TYPEs”, “Ref_frames” and the like.

That is, by using the encoding syntax, it is possible to repeatedlytransmit difference information MVD of a motion vector and a predictedresidual signal data encoding syntax (a Texture Coding Syntax), inresponse to an act of transmitting a “MB_TYPE”, a “Ref_frame” or thelike. Here, the predicted residual signal data encoding syntax isobtained by subjecting a quantized orthogonal transformation coefficient11 to variable length encoding.

FIG. 15 shows an example of a reference frame code (an Ref_frame) basedon a combination of a reference frame picture number and identificationinformation of a generation method of a reference picture.

Here, as shown in FIG. 15, the same reference frame codes (“0” to “4”)as those denoting the reference frame picture numbers of theconventional H. 26L are used for the first reference picture, and newlyadded reference frame codes (“5” to “9”) are used for the secondreference picture.

According to the embodiment, the second reference picture is morestrongly smoothed than the first reference picture which is a normalreference picture, and the second reference picture is a referencepicture which does not have space resolution that an original picture (areference frame picture 5) has.

Thus, the second reference picture is used, only when encodingdistortion is suppressed more when a reference picture of a strongerdegree of smoothness is used and motion compensation efficiency isimproved. Thus, it is less likely that the second reference picture ischosen, compared with the first reference picture.

In many cases, therefore, the first reference picture which is a normalreference picture is chosen as a reference picture used for motioncompensation. A reference frame code table for defining reference framecodes to be transmitted in this case is similar to that for definingreference frame codes (Ref_frames)” of the “H. 26L encoding system”shown in FIG. 16. Thus, compared with the case of the conventional “H.26L encoding system”, there is no increase in a bit amount caused bychange of reference frame codes.

Furthermore, in the case of executing motion compensation which uses thesecond reference picture of a stronger degree of smoothness, arelatively long encoding length is necessary for a reference frame codeto be transmitted. However, a probability of using such a secondreference picture is not large, and an influence of an increased bitamount of the reference frame code may be small compared with motioncompensation efficiency increased by using the second reference pictureof a strong degree of smoothness. Thus, highly efficient encoding can beexpected.

Operations/Effects of the Moving Picture Encoding and Decoding DevicesAccording to the Embodiment 2

According to the moving picture encoding device 20 of the presentinvention, by using two kinds of reference pictures of different degreesof smoothness, i.e., a normal reference picture (a first referencepicture) formed by the reference picture generation section 45 and areference picture (a second reference picture) of a strong degree ofsmoothness, it is possible to carry out motion compensation using areference picture of a degree of smoothness different from onepredetermined area (e.g., a macro-block) to be encoded to another.

Additionally, according to the moving picture encoding device 20 of theinvention, a degree of smoothness for the reference picture can besignaled by generating a reference frame code through combining the“identification information of the method of generating the referencepicture” with the “reference frame picture number”. Accordingly, it ispossible to solve the problem that strong smoothing is always applied tothe “motion compensation value” in the area having a motion vector whichindicates the same pixel position as the “funny position” as in the caseof the “H. 26L encoding system”.

The filter of the present embodiment for generating the second referencepicture which is a reference picture of a strong degree of smoothness isonly an example. By applying a filter for processing other thansmoothing, it is possible to realize prediction based on a referencepicture of a different nature.

Additionally, in FIG. 11, to simplify explanation, the maximum number ofreference frame pictures used for reference picture generation is “5”.However, the present invention is not limited to this number, and amaximum number of reference frame pictures can be optionally set.

According to the “TML-8”, the maximum number of reference frame picturesused for reference picture generation is given as known in the movingpicture encoding device 20 and the moving picture decoding device 50.

Further, in real application, the maximum number of reference framepictures may be decided by such a method or based on information of thecompressed stream 12 sent from the moving picture encoding device 20 tothe moving picture decoding device 50.

In any of the cases, the maximum number of reference frame pictures usedfor reference picture generation is uniquely determined in the movingpicture encoding device 20 and the moving picture decoding device 50.Thus, a reference frame code table can be uniquely determined inaccordance with the maximum number of reference frame pictures used forreference picture generation.

Further, in the reference frame code table for defining reference framecodes (Ref_frames) in FIG. 16, a “reference frame code (an Ref_frame)”which is a combination of the “reference frame picture number” and the“second reference picture” is allocated after a “reference frame code(an Ref_frame)” which is a combination of the “reference frame picturenumber” and the “first reference picture”. However, based on theassumption that reference pictures close together with time for motioncompensation are frequently used, those of smaller reference framepicture numbers among second reference pictures of strong degrees ofsmoothness can be arranged higher in place on the reference frame codetable.

Additionally, the reference frame code table may be determined uniquelyin accordance with the encoding conditions of the predetermined area(the macro-block) to be encoded as described above. Or the referenceframe code may be changed dynamically in accordance with theaforementioned encoding conditions. Here, for the aforementionedencoding conditions, a prediction mode (i.e., kind of a unit fordetecting the motion vector), the quantization step (the QP value) andthe like are conceivable. Here, for the kind of the unit for detectingthe motion vector, for example, a size of a sub-block for detecting themotion vector and the like is conceived.

As a specific example, a case of dynamically changing the referenceframe code in accordance with the quantization step will be described.Here, the second reference picture of a strong degree of smoothness maybe frequently used, when low bit rate encoding is applied. Accordingly,if the quantization step is equal to or lower than a predeterminedthreshold value, the “reference frame code” containing the “secondreference picture” is arranged lower on the reference frame code table.If the quantization step exceeds the predetermined threshold value, some“reference codes” containing “second reference pictures” are arrangedupper on the reference frame code table.

As described above, the “reference frame code table” for defining the“reference frame code” which is a combination of the “reference framepicture number” and the “identification information of the method ofgenerating the reference picture” shown in FIG. 15 is only an example.It is possible to use an given reference frame code table necessary forrealizing switching, to be executed according to the embodiment, betweenthe “reference frame picture number” and the “identification informationof the method of generating the reference picture”.

Additionally, according to the present embodiment, the degree ofsmoothness of the reference picture may be automatically switched to beuniquely determined in accordance with the encoding conditions of thepredetermined area (the macro-block) to be encoded, in place of theexplicit signaling of the degree of smoothness of the reference pictureused for the motion compensation by using the reference frame code (theRef_frame).

That is, the reference picture generation section 45 may generate areference picture 17 of a predetermined degree of smoothness, inaccordance with the encoding conditions (units for detecting the motionvector, quantization steps and the like) of the predetermined area (themacro-block) to be encoded.

For example, complex motion may occur in an area encoded by a“macro-block mode (an MB_Type: a prediction mode)” in which thepredetermined area (the macro-block) to be encoded is finely divided,and accordingly the reference picture used for motion compensation maynot need high pixel value accuracy.

Further, in the macro-block of a large quantization step (a QP value),the reference picture used of motion compensation may not need highpixel value accuracy.

Thus, with regard to the macro-block in which the number of units(sub-blocks) for detecting the motion vector or the quantization stepexceeds a predetermined threshold value, the second reference picture ofa strong degree of smoothness may always be used.

In such a case, since the reference picture to be generated is uniquelydecided in accordance with the encoding conditions, no information foridentifying a degree of smoothness by using the reference frame picturenumber is necessary. Thus, compared with the “H. 26L encoding system”,there is no increase in a bit amount caused by changing of referenceframe codes or macro-block mode codes.

MODIFIED EXAMPLE 2A

Modified Example 2A of the foregoing Embodiment 2 will be described.Hereinafter, differences between the present embodiment and Embodiment 2will be described.

According to the foregoing Embodiment 2, the reference pictures (firstand second reference pictures) 17 of two different kinds of generationmethods (degrees of smoothness) are formed. Subsequently, the formed“identification information (information indicating a degree ofsmoothness) of the method of generating the reference picture” and “thereference frame picture number (identification information of thereference frame picture)” are combined to generate the “reference framecode (the information regarding the reference picture)”. Thus, themotion compensation can be carried out by using the reference picturewhose degree of smoothness is changed from one predetermined area (amacro-block) to be encoded to another.

However, the switching of degrees of smoothness has been possible onlyby the macro-block unit which is a unit for allocating the referenceframe picture number.

Thus, in the modified example, description will be made of a movingpicture encoding device 20 and a moving picture decoding device 50 whichcan transmit a combination of “reference frame code” and a “sub-blockunit (a unit for detecting a motion vector)”. The “reference frame code”is generated by combining a reference frame picture number withidentification information of a method of generating a referencepicture. The sub-block is part of a macro-block in which the motioncompensation is executed.

Basic operations of the moving picture encoding and decoding devices 20and 50 according to the modified example is virtually the same as thoseof the moving picture encoding and decoding devices 20 and 50 accordingto the foregoing Embodiment 2.

According to the modified example, a motion compensation section 33switches reference pictures (a first reference picture or a secondreference picture) 17 used for calculating a motion compensation valueby a unit (a sub-block unit) for detecting a motion vector.

A variable length encoding section 40 transmits a combination of“information regarding reference pictures (Ref_frames)” and “informationindicating motion compensation value (a predicted residual signal dataencoding syntax)” by a unit (a sub-block unit) for detecting a motionvector.

Additionally, a variable length decoding section 71 decodes “informationregarding reference pictures (reference frame codes)” and “informationindicating motion compensation value (a predicted residual signal dataencoding syntax)” by a unit (a sub-block unit) for detecting a motionvector.

Further, a motion compensation section 72 switches the referencepictures 17 used for calculating the motion compensation value by a unit(a sub-block unit) for detecting a motion vector.

Here, as an example of a “reference code (an Ref_frame)” based on acombination of a “reference frame picture number” and “identificationinformation of a method of generating a reference picture,” a codesimilar to that of the Embodiment 2 is assumed.

FIG. 17 shows an encoding syntax based on a macro-block unit accordingto the modified example.

According to the modified example, in the macro-block, a need arises totransmit the “reference frame code (the Ref_frame)” generated bycombining the “reference frame picture number” and the “identificationinformation of the generation method of the reference picture” by asub-block unit by a plurality of times.

The number of “reference frame codes (Ref_frames)” can be notified by a“macro-block type (an MB_type)”, because a type and the number of asub-block are transmitted by the “macro-block type (the MB_type)”.

For example, if a macro-block type is an “INTER prediction mode (a mode7)”, the number of “reference frame codes (Ref_frames)” to betransmitted is “16”.

According to the modified example, in the “sub-block” unit which is aunit for detecting a motion vector in the macro-block, it is possible tocarry out motion compensation by using the reference picture 17 of adifferent degree of smoothness.

Additionally, it is possible to carry out motion compensation which usesa reference frame picture 5 different from sub-block to sub-block.Accordingly, motion compensation of a higher degree of freedom can becarried out in response to a shape and motion of a frame picture signal2.

Furthermore, according to the modified example, the degree of smoothnessof the reference picture may be automatically switched to be uniquelydetermined in accordance with the encoding conditions of thepredetermined area (a macro-block or a sub-block) for detecting a motionvector, in place of the explicit signaling of the degree of smoothnessof the reference picture used for the motion compensation by using thereference frame code (the Ref_frame).

MODIFIED EXAMPLE 2B

The Modified Example 2 of the Embodiment 2 will be described. Except formotion compensation executed by using a different reference picture 17without using a “funny position”, operations of the moving pictureencoding device 20 and the moving picture decoding device 50 accordingto the present embodiment are similar to those of the moving pictureencoding device 20 and the moving picture decoding device 50 describedin the “TML-8”. Thus, details thereof are omitted, and description willfocus on differences.

According to the foregoing Embodiment 2, the reference pictures (firstand second reference pictures) of two different kinds of generationmethods (degrees of smoothness) are formed. Subsequently, the formed“identification information of the generation method of the referencepicture” and “the reference frame picture number” are combined togenerate the “reference frame code”. Thus, the motion compensation inwhich degrees of smoothness is changed from one predetermined area(e.g., a macro-block: a unit for allocating a reference frame picturenumber) to be encoded to another can be carried out.

According to the modified example, two kinds of reference pictures(first and second reference pictures) of different degrees of smoothnessare formed, and the formed “identification information of the generationmethod of the reference picture” and a “macro-block mode” are combinedto generate a “macro-block mode code”. Thus, it is possible to carry outmotion compensation whose degrees of smoothness are changed from onepredetermined area (a macro-block) to be encoded to another.

Basic operations of the moving picture encoding device 20 and the movingpicture decoding devices 50 according to the modified example arevirtually the same as those of the moving picture encoding device 20 andthe moving picture decoding devices 50 according to the foregoingEmbodiment 2.

According to the modified example, “information regarding a referencepicture (a macro-block mode code)” is a combination of “identificationinformation indicating a unit for detecting a motion vector (amacro-block mode: MB_Type)” and “information indicating a degree ofsmoothness (a first reference picture or second reference picture)”.

A variable length encoding section 40 transmits a combination of“information regarding reference picture (macro-block mode codes)”,“identification information of reference frame pictures (a referenceframe picture number: an Ref_frame)” and “information indicating motioncompensation value (a predicted residual signal data encoding syntax)”,by a unit of a predetermined area (a macro-block unit) to be encoded.

Additionally, a variable length decoding section 71 decodes “informationregarding reference pictures (macro-block mode codes: MB_Types)”,“identification information of reference frame pictures (a referenceframe picture number: an Ref_frame)” and “an information predictedresidual signal data encoding syntax indicating motion compensationvalue”, by a unit of a predetermined area (a macro-block) to be encoded.

Further, a motion compensation section 72 uses a reference picture 17specified by “information regarding a reference picture (an MB_Type)”and “identification information of a reference frame picture (anRef_frame)” in place of a reference frame picture 5, so as to calculatea motion compensation value for the predetermined area (the macro-block)to be encoded.

FIG. 14 shows an encoding syntax based on a macro-block unit accordingto the H. 26L encoding system. According to the modified example, thereis no change from the conventional “encoding syntax by the macro-blockunit of the H. 26L encoding system”. However, definition of a“macro-block mode” (an MB_Type)” is changed to be expressed by acombination of a “macro-block mode” and “identification method of ageneration method of a reference picture”.

FIG. 18 shows an example of a “macro-block mode code (an MB_type)” basedon a combination of a “macro-block mode” and “identification informationof a method of generating a reference picture”.

As shown in FIG. 18, with regard to the same macro-block mode as themacro-block mode of the conventional H. 26L, execution of motioncompensation is instructed from a first reference picture. With regardto a newly added macro-block mode, execution of motion compensation isinstructed from a second reference picture of a strong degree ofsmoothness.

According to the modified example, since the macro-block mode isallocated to a macro-block unit, it is possible to carry out motioncompensation from a reference picture of a different degree ofsmoothness by a macro-block unit.

Furthermore, according to the modified example, the degree of smoothnessof the reference picture may be automatically switched to be uniquelydetermined in accordance with the encoding conditions of an area (amacro-block) to be encoded, in place of the explicit signaling of thedegree of smoothness of the reference picture used for the motioncompensation by using the macro-block mode code (the MB_Type).

MODIFIED EXAMPLE 2C

The Modified Example 2C of the foregoing Embodiment 2 will be described.According to the modified example, the second reference picture of theforgoing Embodiment 2 is subjected to strong smoothing, and a motionvector of the same accuracy as that of the first reference picture isnot always necessary in motion compensation. Thus, description will bemade of a configuration in which accuracy of a motion vector is changedwhen the second reference picture is used. Hereinafter, differencesbetween the modified example and the foregoing embodiment 2 will bedescribed.

According to the modified example, when a reference picture (a secondreference picture) of a strong degree of smoothness is used, a motioncompensation section 33 reduces accuracy of a motion vector used forcalculating a motion compensation value (a predicted picture signal 6).Specifically, in such a case, the motion compensation section 33 isconfigured to reduce accuracy of horizontal and vertical elements of themotion vector.

According to the modified example, as in the case of the foregoingEmbodiment 2, a reference picture generation section 45 generates areference picture subjected to strong smoothing as a second referencepicture in which space resolution is reduced to ½ pixel accuracy orinteger pixel accuracy, after generating a first reference picture.

As an example of a method of generating a reference picture subjected tostrong smoothing in a second reference picture in which space resolutionis ½ pixel accuracy, a method of down-sampling by operating a (1, 2,1)/4 filter on a first reference picture will be described.

Additionally, as an example of a method of generating a referencepicture subjected to strong smoothing in which space resolution isinteger pixel accuracy, a method of down-sampling by operating a (1, 2,1)/4 filter on the smoothed picture of ½ pixel accuracy will bedescribed.

According to the modified example, as an example of a “reference framecode” based on a combination of a “reference frame picture number” and“identification information of a method of generating a referencepicture”, a code similar to that of the foregoing Embodiment 2 isassumed.

For example, when a strongly smoothed reference picture whose spaceresolution is reduced to ½ pixel accuracy is generated as a secondreference picture, the following substitution is carried out in a motionvector of the second reference picture assuming that a motion vector ofa first reference picture is (MVx, MVy).

Pixel position of the first reference picture:

-   -   MVx, MVy (a unit is ¼ pixel)

Pixel position of the second reference picture:

-   -   MVx//2, MVy//2

Here, “//” represents integer division which accompanies a roundingoperation in a zero direction.

Additionally, if a strongly smoothed reference picture whose spaceresolution is reduced to integer pixel accuracy is generated as a secondreference picture, the following substitution is carried out in a motionvector of the second reference picture assuming that a motion vector ofa first reference picture is (MVx, MVy).

Pixel position of the first reference picture:

-   -   MVx, MVy (a unit is ¼ pixel)

Pixel position of the second reference picture:

-   -   MVx//4, MVy//4

Here, “//” represents integer division which accompanies a roundingoperation in a zero direction.

Thus, the second reference picture has a plurality of values for thesame motion vector MV=(MVx, MVy).

For example, if the second reference picture is reduced to ½ pixelaccuracy, (3, 3), (2, 3), (3, 2) and (2, 2) indicate the same motionvector.

Therefore, in the second reference picture, when encoding is executed atthe variable length encoding section 40, a motion vector (e.g., (2, 2))in which a generated encoding amount is small may be sent as arepresentative.

Instead, in the second reference picture, difference information MVDbetween a predicted motion vector PMV and a motion vector may becalculated by using a motion vector after substitution is executed inaccordance with space resolution, and a value of the differenceinformation of the motion vector to be sent may be reduced. In otherwords, an encoding amount may be reduced.

In this case, if the difference information MVD between the predictedmotion vector and the motion vector is calculated from the secondreference picture to the first reference picture of large spaceresolution, substitution is inversely carried out to increase the spaceresolution of the motion vector MV of the second reference picture.

By the aforementioned change, if noise is superimposed on the referenceframe picture 5, efficient encoding can be expected by referring to thesecond reference picture of low space resolution.

Additionally, the modified example is characterized in that in thesecond reference picture of low space resolution, the space resolutionof the motion vector is reduced, and redundancy at the time of motionvector encoding is avoided.

Furthermore, according to the modified example, the space resolution ofthe reference picture may be automatically switched to be uniquelydetermined in accordance with the encoding conditions of an area (amacro-block) to be encoded, in place of the explicit signaling of thespace resolution of the reference picture used for the motioncompensation by using the reference frame code (the Ref_frame).

For example, in an area encoded with a “macro-block mode (an MB_Type: aprediction mode)” in which the predetermined area (the macro-block) tobe encoded is finely divided, complex motion may occur, and accordinglythe reference picture used for motion compensation may not need highpixel value accuracy.

Additionally, in a macro-block of a large quantization step (a QPvalue), the reference picture used for motion compensation may not needhigh pixel value accuracy.

Thus, with regard to the macro-block in which the number of units(sub-blocks) for detecting a motion vector or the quantization stepexceeds a predetermined threshold value, a second reference picture ofstrong smoothing (i.e., low space resolution) may always be used.

In such a case, since the generated reference picture is uniquelydetermined in accordance with the encoding conditions, no informationfor identifying a degree of smoothness by using a reference framepicture number is necessary. Accordingly, compared with the “H. 26Lencoding system”, there is no increase in a bit amount caused by achange of a reference frame code or a macro-block mode code.

Further, the degree of smoothness and the space resolution can beautomatically switched to be uniquely determined in accordance with theaforementioned encoding conditions (macro-block modes, quantizationsteps or the like).

In such a case, with regard to the macro-block in which the dividednumber of sub-blocks or the quantization step exceeds a predeterminedthreshold value, an encoding amount of the generated motion vector isreduced, by reducing space resolution to ½ pixel accuracy or integerpixel accuracy, by always utilizing a reference picture which uses afiler to apply a strong degree of smoothness, and by decreasing pixelaccuracy of horizontal and vertical elements of the motion vector.

(Embodiment 3)

Description will be made of a moving picture encoding device 20 and amoving picture decoding device 50 according to the Embodiment 3.Description of the Embodiment 2 has referred to the moving pictureencoding device 20 and the moving picture decoding device 50. The movingpicture encoding and decoding devices enable motion compensation to beexecuted by using the normal reference picture (the first pictureinformation) and the strongly smoothed reference picture (the secondpicture information) for each predetermined area (the macro-block) to beencoded, by signaling the “information regarding a reference picture(the reference frame code or macro-block mode code)” which is acombination of the “identification information of a method of generatingthe reference picture (the first reference picture or second referencepicture)” and the “reference frame picture number”.

In the present embodiment, description will be made of the movingpicture encoding device 20 and the moving picture decoding device 50which constitute a pyramid of layers for each space resolution, so as toenable reference pictures having three kinds of different pixel accuracyto be generated.

FIG. 19 shows a schematic configuration of the moving picture encodingdevice 20 of the embodiment, and FIG. 20 shows a schematic configurationof the moving picture decoding device 50.

Basic operations of the moving picture encoding device 20 and the movingpicture decoding device 50 according to the present embodiment isvirtually the same as those of the moving picture encoding device 20 andthe moving picture decoding device 50 according to the conventionaltechnology, except that modifications have been introduced inconfiguration of the motion detection section 32, the motioncompensation sections 33 and 72 as well as the variable length encodingsection 40, and except that hierarchical reference picture generationsections 46 and 81 have been added.

According to the present embodiment, each of the hierarchical referencepicture generation sections 46 and 81 constitutes a reference picturegeneration section configured to generate a plurality of differentreference pictures (hierarchical reference pictures 18), by executing aplurality of different picture processing on a reference frame picture15. Here, as the aforementioned picture processing, processing forchanging degrees of smoothness, processing for changing spaceresolution, and the like are conceived. Description of the embodimentwill refer to the processing for changing the space resolution is usedas the aforementioned picture processing.

Additionally, each of the hierarchical reference picture generationsections 46 and 81 generates a reference picture (a hierarchicalreference picture 18) having a plurality of space resolutions byexecuting filtering through a filter which has a plurality of differentpass bands. Here, “information indicating picture processing, i.e.,information indicating space resolution (layer)” is an identifier of thefilter.

Further, the motion detection section 32 constitutes a 3-dimensionalmotion vector generation section configured to generate a “3-dimensionalmotion vector (Layer, MVx, MVy)” by correlating a “motion vector (MVx,MVy)” detected by using the reference picture (the hierarchicalreference picture 18) with “information indicating space resolution ofthe reference picture (the hierarchical reference picture 18) (Layer)”.

The motion detection section 32 may be configured to reduce accuracy ofthe 3-dimensional motion vector for a reference picture of low spaceresolution (e.g., a layer 3 or the like).

Additionally, the motion compensation section 33 (72) is configured tocalculate a motion compensation value for a predetermined area (amacro-block) to be encoded (decoded), by using the reference picture(the hierarchical reference picture 18) in place of the reference framepicture 5.

Further, the motion compensation section 33 constitutes a 3-dimensionalmotion vector prediction section configured to predict a 3-dimensionalmotion vector, by using a correlation (e.g., a switching of a context inarithmetic encoding) between an encoded predetermined area (an encodedmacro-block) in a frame picture and the predetermined area to be encoded(the macro-block to be encoded).

The variable length encoding section 40 constitutes a transmissionsection configured to transmit a combination of the “3-dimensionalmotion vector” and “information indicating a motion compensation value”.

Incidentally, the variable length encoding section 40 may transmit acombination of difference information (Layer D, MVDx, MVDy) between the3-dimensional motion vector

(Layer, MVx, MVy) generated by the motion detection section 32 and a3-dimensional motion vector (Player, PMVx, PMVy) predicted by the motioncompensation section 33, and information indicating a motioncompensation value.

Additionally, the variable length decoding section 71 constitutes adecoding section configured to decode a 3-dimensional motion vector of apredetermined area to be decoded.

First, a concept used in the present embodiment will be described withreference to FIGS. 21 and 22.

Each of the hierarchical reference picture generation sections 46 and 81generates 3 layers for the reference frame picture 5 used for the motioncompensation by each of the motion compensation sections 33 and 72.

First, as shown in FIGS. 21 and 22, each of the hierarchical referencepicture generation sections 46 and 81 subjects the reference framepicture 5 to up-sampling by 8 tap filers, so as to generate a layer 1 of¼ pixel accuracy which is one of hierarchical reference pictures 18.Examples of the 8 tap filters used here are as follows:

-   -   for ¼ pixel position:

(−3, 12, −37, 229, 71, −21, 6, −1)/256

-   -   for 2/4 pixel position:

(−3, 12, −39, 158, 158, −39, 12, −3)/256

-   -   for ¾ pixel position:

(−1, 6, −21, 71, 229, −37, 12, −3)/256

Here, with regard to a pixel value of an integer pixel position, a pixelvalue of the same position of the reference frame picture 5 is copied.Pixel values of the ¼ pixel position, the 2/4 pixel position and the ¾pixel position between integer pixel positions are found bymultiplication and summation of the aforementioned filter coefficientsfor the pixel value of the integer pixel position. This filterprocessing is carried out separately in horizontal and verticaldirections.

The filtering processing is described in an up-sampling operation of ⅛pixel accuracy of the conventional “TML-8”, and thus details thereof areomitted.

Second, each of the hierarchical reference picture generation sections46 and 81 subjects the generated layer 1 of ¼ pixel accuracy todown-sampling by 3 tap filters (low pass band type filters), so as togenerate a layer 2 of ½ pixel accuracy which is one of hierarchicalreference pictures 18. An example of the 3 tap filters used here is “(1,2, 1)/4.”

Third, each of the hierarchical reference picture generation sections 46and 81 subjects the generated layer 2 of ½ pixel accuracy todown-sampling by the 3 tap filters, so as to generate a layer 3 ofinteger pixel accuracy which is one of hierarchical reference pictures18. The 3 tap filers used here are the same as those which have beenpreviously used.

Noted that the layer 1 has ¼ pixel accuracy as in the case of theconventional technology, but a pixel value of the ¼ pixel position iscalculated not by linear interpolation but by executing theaforementioned filter processing so as to maintain space resolution ofan original picture (an reference frame picture 5).

As described above, each of the hierarchical reference picturegeneration sections 46 and 81 generates reference pictures (layer 1 tolayer 3) having a plurality of space resolutions through filterprocessing by the filter having the plurality of different bass bands.

According to the present embodiment, the motion compensation section 33executes motion compensation by using the hierarchical reference picture18 generated in the foregoing manner.

In this event, a motion vector 3 is not a group of 2 terms (a2-dimensional motion vector) of (MVx, MVy) but a group of 3 terms (a3-dimensional motion vector) of (Layer, MVx, MVy).

The motion detection section 32 detects the 3-dimensional motion vector(Layer, MVx, MVy) in place of the 2-dimensional motion vector (MVx,MVy).

The layer 2 has space resolution half of that of the layer 1, andfurther the layer 3 has space resolution half of that of the layer 2.Thus, the following substitution is carried out.

Pixel position of layer 1: MVx, MVy (a unit is ¼ pixel)

Pixel position of layer 2: MVx//2, MVy//2

Pixel position of layer 3: MVx//4, MVy//4

Here, “//” represents integer division accompanied by a roundingoperation in a zero direction.

Thus, the layers 2 and 3 have a plurality of values (MVx, MVy) for thesame motion vector.

For example, in the layers 2 and 3, (2, 3, 3), (2, 2, 3), (2, 2, 3) and(2, 2, 2) indicate the same motion vector.

Therefore, in the upper layer (a layer 2 or layer 3), when encoding isexecuted at the variable length encoding section 40, a motion vector ofa small amount of encoding (e.g., (2, 2, 2) in the aforementioned case)may be sent as a representative.

Instead, in the layers 2 and 3, difference information MVD between apredicted motion vector PMV and a motion vector may be calculated byusing the motion vector after substitution in accordance with spaceresolution of each layer, and a value of the difference information MVDof the motion vector to be sent may be reduced. That is, an amount ofencoding may be reduced.

In this case, when the difference information MVD between the predictedmotion vector PMV of the motion vector of large space resolution of thelayer 1 and the motion vector is calculated from such a layer,substitution is conversely executed to increase the space resolution ofthe motion vector of each layer.

By using the above concept, as shown in FIG. 23, the predicted motionvector calculation section 33 c of the motion compensation section 22predicts a 3-dimensional motion vector expanded to a plurality of spaceresolutions, by a method that is virtually the same as that of theforegoing Embodiment 1.

In FIG. 23, a “PMVx_(E)” denotes a horizontal element of a predictedmotion vector PMV_(E) for a predetermined area (a macro-block) E to beencoded, and a “PMVy_(E)” denotes a vertical element of the predictedmotion vector PMV_(E) for the predetermined area (the macro-block) E tobe encoded.

A “Player_(E)” indicates predicted space resolution of the predictedarea (the macro-block) E to be encoded.

Each of the “MVDx_(E)” and the “MVDy_(E)” indicates differenceinformation of a motion vector of ¼ pixel accuracy for the predeterminedarea (the macro-block) E to be encoded. A “LayerD” indicates differenceinformation of space resolution for the predetermined area (themacro-block) E to be encoded.

Thus, encoding processing is carried out for a group of 3 terms of(LayerD, MVDx, MVDy).

Incidentally, the motion compensation section 33 may predict a3-dimensional motion vector of a predetermined area (a macro-block) tobe encoded, by using the 3-dimensional motion vector of the encodedpredetermined area (the macro-block) in the frame picture signal 2,calculate difference information between the predicted 3-dimensionalmotion vector and the 3-dimensional motion vector detected by the motionvector detection section 32, and execute motion compensation by usingthe calculated difference information of the 3-dimensional motionvector.

Here, it is normally predicted if the LayerD focuses on zero, and itstransition is asymmetrical when seen from the layer 1 or the layer 3.

Thus, according to the present embodiment, “adaptive arithmeticencoding” realized in the conventional technology may be used, andfurther expansion in which 3 states of a context model of Player areadded may be executed so as to use a correlation with a macro-block of aneighboring area.

As described above, for the Layer, it is possible to carry out encodingwhich uses the correlation with the macro-block of the neighboring area.

In the foregoing description, the predicted difference encoding whichuses the Player and the LayerD is carried out. However, the execution ofthe predicted difference encoding is identical to execution of contextmodeling.

Thus, by using the correlation with the macro-block of the neighboringarea directly or by using an intermediate value such as Player (usingcontext switching) as a context model, not the LayerD but the Layeritself may be encoded by using arithmetic encoding.

Incidentally, since a concept and details of the “adaptive arithmeticencoding” are heretofore described as “Context-based Adaptive BinaryArithmetic Coding”, details thereof are omitted.

Similarly to the foregoing Embodiments 1 and 2 and the modified examplesthereof, the present invention has referred to the new motioncompensation in which the “2-dimensional motion vector” is expanded asthe “3-dimensional motion vector” which contains the “information(Layer) indicating the picture processing (space resolution)”, insteadof transmitting the “2-dimensional motion vector” and the “informationindicating picture processing (information indicating spaceresolution)”.

By referring to FIG. 24, description will be made of a motioncompensation operation in the moving picture encoding device 20 of thepresent embodiment.

In step 1201, the hierarchical reference picture generation section 46generates a hierarchical reference picture 18 by using a reference framepicture 5 extracted from the frame memory 34.

In step 1202, the motion detection section 32 detects a 3-dimensionalmotion vector of a predetermined area (a macro-block) to be encoded, byreferring to the hierarchical reference picture 18 from the hierarchicalreference picture generation section 46.

In step 1203, the motion compensation section 33 generates a predictedpicture signal 6 based on the 3-dimensional motion vector from themotion detection section 32 and the hierarchical reference picture 18from the hierarchical reference picture generation section 46.

According to the present embodiment, if noise is superimposed on thereference frame picture 5, efficient encoding is expected by referringto a layer picture of adaptive low space resolution.

Additionally, the layer picture (e.g., layer 2 or 3) of low spaceresolution is characterized in that space resolution of the3-dimensional motion vector is reduced, and in that redundancy duringencoding of the 3-dimensional motion vector is avoided.

Furthermore, according to the present embodiment, the 3-dimensionalvector is employed. A vector distribution on parameter space of the3-dimensional motion vector is expected to be continuous spatially, andefficiency improvement of encoding can be expected.

The method of generating the predicted picture signal or the motioncompensation value according to the present embodiment is only anexample. It is possible to use an optional generation method necessaryfor realizing the switching of a calculation method of a motioncompensation value executed in the present embodiment.

Operations/Effects of the Moving Picture Encoding and Decoding DevicesAccording to the Embodiment 3

According to the moving picture encoding device according to the presentinvention, since the motion detection section 32 generates a3-dimensional motion vector in accordance with the hierarchicalreference picture 17, it is possible to carry out motion compensation ofpixel accuracy different from one predetermined area to be encoded toanother.

MODIFIED EXAMPLE 3A

The Modified Example 3A of the foregoing Embodiment 3 will be described.Hereinafter, differences between the modified example and the Embodiment3 will be described by referring to FIG. 25.

First, the modified example is different from the foregoing Embodiment 3in that motion compensation is executed not by ¼ pixel accuracy, but by½ pixel accuracy.

That is, according to the modified example, each of the hierarchicalreference picture generation section 46 and 81 executes up-sampling byapplying 6 tap filters (1, −5, 20, 20, −5, 1)/32″ used in theconventional “TML-8” to the reference frame picture 5, so as to generatea layer 1 of ½ pixel accuracy.

Second, the modified example is different from the foregoing Embodiment3 in that each of the hierarchical reference picture generation section46 and 81 independently applies a smoothing filter “(1, 2, 1)/4” to thelayer 1 horizontally/vertically, so as to generate a layer 2.

Third, the modified example is different from the foregoing Embodiment 3in that the number of layers is 2 (layer 1 and layer 2), and in thatboth has the same space resolution.

Thus, the followings are realized:

Pixel position of layer 1: MVx, MVy (a unit is ½ pixel)

Pixel position of layer 2: MVx, MVy (a unit is ½ pixel)

By adding the aforementioned changes and executing “intermediate valueprediction” based on the same method as that of the conventionaltechnology, the 3-dimensional motion vector of (LayerD, MVDx, MVDy) isencoded. “Adaptive arithmetic encoding” is different from that of theforegoing Embodiment 3 in that a context of PlayerE takes two states.

By the aforementioned changes, while motion compensation is executed by½ pixel accuracy, if noise is superimposed on the reference framepicture 5, it is possible to carry out the motion compensation byswitching to adaptive low resolution.

Especially, in low rate encoding, it is assumed that ¼ pixel accuracy isunnecessary, and rather there is a tendency to use a smoothed picture ofmotion compensation of ¼ pixel accuracy. Thus, here, a system ofswitching to an picture of explicitly low space resolution is describedas the modified example.

Incidentally, according to the modified example, the filter “(1, 2,1)/4” for generating the layer 2 picture is a simple low pass typefilter. However, a smoothing filter of an edge holding type may be used.

For example, among smoothing filters of the edge holding type are a“median filter” for obtaining an intermediate value of an area of 3pixels×3 pixels, and a “dynamic weighting filter” described in U.S. Pat.No. 6,041,145 “Device and method for smoothing picture signal, deviceand method for encoding picture and device and method for decodingpicture” can be used.

The aforementioned “dynamic weighting filter” is what executes adaptivesmoothing by calculating a difference absolute value between a smoothedcenter pixel value and its neighboring pixel value, and by providing afilter coefficient inversely proportional to the difference absolutevalue to a peripheral pixel value (near 8).

Incidentally, a program for causing a computer 100 to function as themoving picture encoding device 20 or the moving picture decoding device50 of the present invention can be stored in a computer readablerecording medium.

As the computer readable recording medium, as shown in FIG. 26, forexample, a floppy disk 101, a compact disk 102, an IC chip 103, acassette tape 104 or the like can be listed. According to such acomputer readable recording medium which stores the program, theaforementioned program can be easily saved, transported, sold or thelike.

INDUSTRIAL APPLICABILITY

As described above, according to the present invention, it is possibleto express a predicted picture signal with light overheads, and toprovide motion compensation of different pixel accuracy.

The invention claimed is:
 1. A moving picture encoding device forencoding a moving picture constituted of a time sequence of framepictures by motion compensation, the device comprising: a referencepicture generation section configured to generate a plurality ofdifferent reference pictures, by executing a plurality of differentpicture processing on a single reference frame picture, in accordancewith an encoding condition of a predetermined area to be encoded,wherein the reference picture generation section is configured togenerate, from the single reference frame picture, a first referencepicture that has no funny position, and a second reference picture, fromthe single reference frame, that is more strongly smoothed than thefirst reference picture; and a motion compensation section configured tocalculate a motion compensation value for the predetermined area to beencoded, by using a generated reference picture subjected to thepredetermined picture processing, wherein the moving picture encodingdevice implements an image processing using the plurality of differentreference pictures, the motion compensation section determines a motioncompensation value of the first reference picture that is equal to amotion compensation value in each pixel position of an integer pixelposition, a ½ pixel position and a ¼ pixel position of a block definedin TML-8, except that an original motion compensation value is used fora motion compensation value in a same position as the funny position ofthe block defined in TML-8, and the motion compensation section uses thesecond reference picture to calculate the motion compensation value onlywhen distortion is suppressed more than when using the first referencepicture.
 2. The moving picture encoding device according to claim 1,wherein the reference picture generation section is configured togenerate a reference picture in accordance with a prediction mode. 3.The moving picture encoding device according to claim 1, wherein thereference picture generation section is configured to generate areference picture in accordance with a quantization processing.
 4. Amoving picture decoding device for decoding a moving picture constitutedof a time sequence of frame pictures by motion compensation, the devicecomprising: a reference picture generation section configured togenerate a plurality of reference pictures from a single reference framepicture, in accordance with an encoding condition of a predeterminedarea to be decoded, wherein the reference picture generation section isconfigured to generate, from the single reference frame picture, a firstreference picture that has no funny position, and a second referencepicture, from the single reference frame, that is more strongly smoothedthan the first reference picture; and a motion compensation sectionconfigured to calculate a motion compensation value for thepredetermined area to be decoded, by using a generated reference picturesubjected to the predetermined picture processing, wherein the movingpicture decoding device implements an image processing using theplurality of different reference pictures, the motion compensationsection determines a motion compensation value of the first referencepicture that is equal to a motion compensation value in each pixelposition of an integer pixel position, a ½ pixel position and a ¼ pixelposition of a block defined in TML-8, except that an original motioncompensation value is used for a motion compensation value in a sameposition as the funny position of the block defined in TML-8, and themotion compensation section uses the second reference picture tocalculate the motion compensation value only when distortion issuppressed more than when using the first reference picture.
 5. Themoving picture decoding device according to claim 4, wherein thereference picture generation section is configured to generate areference picture in accordance with a prediction mode.
 6. The movingpicture decoding device according to claim 4, wherein the referencepicture generation section is configured to generate a reference picturein accordance with a quantization processing.
 7. A moving pictureencoding method for encoding a moving picture constituted of a timesequence of frame pictures by motion compensation, the methodcomprising: generating a plurality reference pictures from a singlereference frame picture, in accordance with an encoding condition of apredetermined area to be encoded, wherein the generating includesgenerating, from the single reference frame picture, a first referencepicture that has no funny position, and a second reference picture, fromthe single reference frame, that is more strongly smoothed than thefirst reference picture; calculating a motion compensation value for thepredetermined area to be encoded, by using a generated reference picturesubjected to the predetermined picture processing, wherein thecalculating includes determining a motion compensation value of thefirst reference picture that is equal to a motion compensation value ineach pixel position of an integer pixel position, a ½ pixel position anda ¼ pixel position of a block defined in TML-8, except that an originalmotion compensation value is used for a motion compensation value in asame position as the funny position of the block defined in TML-8, andthe calculating uses the second reference picture to calculate themotion compensation value only when distortion is suppressed more thanwhen using the first reference picture; and implementing an imageprocessing using the plurality of different reference pictures.
 8. Themoving picture encoding method according to claim 7, wherein, in thegenerating, a reference picture is generated in accordance with aprediction mode.
 9. The moving picture encoding method according toclaim 7, wherein, in the generating, a reference picture is generated inaccordance with a quantization processing.
 10. A moving picture decodingmethod for decoding a moving picture constituted of a time sequence offrame pictures by motion compensation, the method comprising: generatinga plurality of reference pictures from a single reference frame picture,in accordance with an encoding condition of a predetermined area to bedecoded, wherein the generating includes generating, from the singlereference frame picture, a first reference picture that has no funnyposition, and a second reference picture, from the single referenceframe, that is more strongly smoothed than the first reference picture;calculating a motion compensation value for the predetermined area to bedecoded, by using a generated reference picture subjected to thepredetermined picture processing, wherein the calculating includesdetermining a motion compensation value of the first reference picturethat is equal to a motion compensation value in each pixel position ofan integer pixel position, a ½ pixel position and a ¼ pixel position ofa block defined in TML-8, except that an original motion compensationvalue is used for a motion compensation value in a same position as thefunny position of the block defined in TML-8, and the calculating usesthe second reference picture to calculate the motion compensation valueonly when distortion is suppressed more than when using the firstreference picture; and implementing an image processing using theplurality of different reference pictures.
 11. The moving picturedecoding method according to claim 10, wherein, in the generating, areference picture is generated in accordance with a prediction mode. 12.The moving picture decoding method according to claim 10, wherein, inthe generating, a reference picture is generated in accordance with aquantization process.
 13. A non-transitory computer readable mediumencoded with computer executable instructions for causing a computer tofunction as a moving picture encoding device for encoding a movingpicture constituted of a time sequence of frame pictures by motioncompensation, said computer executable instructions causing the computerto implement a method comprising: generating a plurality of referencepictures from a single reference frame picture, in accordance with anencoding condition of a predetermined area to be encoded, wherein thegenerating includes generating, from the single reference frame picture,a first reference picture that has no funny position, and a secondreference picture, from the single reference frame, that is morestrongly smoothed than the first reference picture; calculating a motioncompensation value for the predetermined area to be encoded, by using agenerated reference picture subjected to the predetermined pictureprocessing, wherein the calculating includes determining a motioncompensation value of the first reference picture that is equal to amotion compensation value in each pixel position of an integer pixelposition, a ½ pixel position and a ¼ pixel position of a block definedin TML-8, except that an original motion compensation value is used fora motion compensation value in a same position as the funny position ofthe block defined in TML-8, and the calculating uses the secondreference picture to calculate the motion compensation value only whendistortion is suppressed more than when using the first referencepicture; and implementing an image processing using the plurality ofdifferent reference pictures.
 14. The non-transitory computer readablemedium according to claim 13, wherein the generating includes generatinga reference picture in accordance with a prediction mode.
 15. Thenon-transitory computer readable medium according to claim 13, whereinthe generating includes generating a reference picture in accordancewith a quantization processing.
 16. A non-transitory computer readablemedium encoded with computer executable instructions for causing acomputer to function as a moving picture decoding device for decoding amoving picture constituted of a time sequence of frame pictures bymotion compensation, said computer readable instructions causing thecomputer to implement a method comprising: generating a plurality ofreference pictures from a single reference frame picture, in accordancewith an encoding condition of a predetermined area to be decoded,wherein the generating includes generating, from the single referenceframe picture, a first reference picture that has no funny position, anda second reference picture, from the single reference frame, that ismore strongly smoothed than the first reference picture; calculating amotion compensation value for the predetermined area to be decoded, byusing a generated reference picture subjected to the predeterminedpicture processing, wherein the calculating includes determining amotion compensation value of the first reference picture that is equalto a motion compensation value in each pixel position of an integerpixel position, a ½ pixel position and a ¼ pixel position of a blockdefined in TML-8, except that an original motion compensation value isused for a motion compensation value in a same position as the funnyposition of the block defined in TML-8, and the calculating uses thesecond reference picture to calculate the motion compensation value onlywhen distortion is suppressed more than when using the first referencepicture; and implementing an image processing using the plurality ofdifferent reference pictures.
 17. The non-transitory computer readablemedium according to claim 16, wherein the generating includes generatinga reference picture in accordance with a prediction mode.
 18. Thenon-transitory computer readable medium according to claim 16, whereinthe generating includes generating a reference picture in accordancewith a quantization processing.